2024.11.04 22:38:52.651308 [ 580 ] {} <Warning> Context: ThreadFuzzer is enabled. Application will run slowly and unstable.
2024.11.04 22:38:52.651404 [ 580 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.11.04 22:38:52.669884 [ 580 ] {} <Warning> Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl.
2024.11.04 22:38:52.671072 [ 580 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.11.04 22:39:08.195291 [ 580 ] {} <Warning> CloudPlacementInfo: Placement info has not been loaded
2024.11.04 22:39:08.797139 [ 580 ] {} <Warning> KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper.
2024.11.04 22:39:08.798063 [ 580 ] {} <Warning> KeeperLogStore: Removing all changelogs
2024.11.04 22:39:08.799518 [ 580 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.11.04 22:39:08.803743 [ 580 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.11.04 22:39:08.803804 [ 580 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.11.04 22:39:08.854784 [ 1230 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.11.04 22:39:09.094838 [ 580 ] {} <Warning> Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist
2024.11.04 22:39:09.094925 [ 580 ] {} <Warning> Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/
2024.11.04 22:39:10.081530 [ 580 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.11.04 22:40:02.289845 [ 1410 ] {} <Error> ZooKeeperClient: Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<unsigned long&, String>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<String>::type>, unsigned long&, String&&) @ 0x0000000008f26d79
3. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x0000000008f269cd
4. ./src/Common/transformEndianness.h:18: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c1569eb
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
9. ? @ 0x00007efd1fa45ac3
10. ? @ 0x00007efd1fad7850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:40:02.549291 [ 1409 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:49594 -> [::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: Coordination::ZooKeeper::flushWriteBuffer() @ 0x000000003c13cefe
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:709: Coordination::ZooKeeper::sendThread() @ 0x000000003c153713
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b252
7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
9. ? @ 0x00007efd1fa45ac3
10. ? @ 0x00007efd1fad7850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:40:02.673844 [ 583 ] {da5a893e-58ca-40c8-aa8e-a08955a6d244} <Warning> InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 <table definition>; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/' <table definition>; to create new table and attach data.
2024.11.04 22:40:03.155385 [ 1410 ] {} <Error> ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:0: Poco::IOException::IOException(String const&, int) @ 0x000000003e9ff82e
2. ./build_docker/./base/poco/Net/src/NetException.cpp:0: Poco::Net::NetException::NetException(String const&, int) @ 0x000000003e668a8e
3. ./build_docker/./base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x000000003e685f7a
4. ./contrib/llvm-project/libcxx/include/string:1499: Poco::Net::SocketImpl::shutdown() @ 0x000000003e68aeb9
5. ./build_docker/./base/poco/Net/src/StreamSocket.cpp:109: Poco::Net::StreamSocket::shutdown() @ 0x000000003e695dee
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1040: Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000003c13ebb8
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c1563d1
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007efd1fa45ac3
12. ? @ 0x00007efd1fad7850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:40:11.685292 [ 583 ] {23e8c3fe-75a0-4efb-b217-4c7071b5efef} <Warning> InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1 <table definition>; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/' <table definition>; to create new table and attach data.
2024.11.04 22:40:19.476208 [ 1231 ] {} <Error> RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled
2024.11.04 22:40:22.351069 [ 1472 ] {} <Warning> Context: ThreadFuzzer is enabled. Application will run slowly and unstable.
2024.11.04 22:40:22.351158 [ 1472 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.11.04 22:40:22.371124 [ 1472 ] {} <Warning> Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl.
2024.11.04 22:40:22.372288 [ 1472 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.11.04 22:40:45.452139 [ 1472 ] {} <Warning> CloudPlacementInfo: Placement info has not been loaded
2024.11.04 22:40:45.913326 [ 1472 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.11.04 22:40:45.919687 [ 1472 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.11.04 22:40:45.919790 [ 1472 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.11.04 22:40:45.971402 [ 2151 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.11.04 22:40:48.387261 [ 1472 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.11.04 22:40:52.874888 [ 2179 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_6_1}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:41:47.168824 [ 2317 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./build_docker/./src/Compression/CompressedReadBufferBase.cpp:163: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x00000000316e6594
6. ./build_docker/./src/Compression/CompressedReadBuffer.cpp:12: non-virtual thunk to DB::CompressedReadBuffer::nextImpl() @ 0x00000000316e59cc
7. DB::ReadBuffer::next() @ 0x0000000008ec3eef
8. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x0000000008f2670b
9. ./src/Common/transformEndianness.h:18: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c1569eb
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
11. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fcab693eac3
15. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:41:47.374488 [ 2319 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:57262 -> [::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: Coordination::ZooKeeper::flushWriteBuffer() @ 0x000000003c13cefe
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:709: Coordination::ZooKeeper::sendThread() @ 0x000000003c153713
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b252
7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
9. ? @ 0x00007fcab693eac3
10. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:41:47.639321 [ 2317 ] {} <Error> ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:0: Poco::IOException::IOException(String const&, int) @ 0x000000003e9ff82e
2. ./build_docker/./base/poco/Net/src/NetException.cpp:0: Poco::Net::NetException::NetException(String const&, int) @ 0x000000003e668a8e
3. ./build_docker/./base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x000000003e685f7a
4. ./contrib/llvm-project/libcxx/include/string:1499: Poco::Net::SocketImpl::shutdown() @ 0x000000003e68aeb9
5. ./build_docker/./base/poco/Net/src/StreamSocket.cpp:109: Poco::Net::StreamSocket::shutdown() @ 0x000000003e695dee
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1040: Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000003c13ebb8
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c1563d1
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fcab693eac3
12. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:45:25.108544 [ 2177 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_1_16_3}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:48:49.256286 [ 2176 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_1_40_9}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:52:59.219747 [ 2181 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_1_72_18}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:53:50.964728 [ 2179 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_186_37}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:56:00.656890 [ 2175 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_215_51}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:57:18.436180 [ 2188 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_235_56}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 22:59:24.055170 [ 2187 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_264_69}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:00:56.245252 [ 2185 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_1_131_30}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fcab693eac3
17. ? @ 0x00007fcab69d0850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:02:16.903664 [ 2174 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded soft memory limit (13.77 GiB)
2024.11.04 23:03:16.907180 [ 2174 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded hard memory limit (14.54 GiB)
2024.11.04 23:03:16.907316 [ 2174 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded soft memory limit (13.77 GiB)
2024.11.04 23:03:18.811715 [ 1475 ] {30b599b5-4c7a-438e-8b65-1f0946d96df5} <Error> executeQuery: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 15.49 GiB (attempt to allocate chunk of 2097152 bytes), maximum: 15.30 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker. (MEMORY_LIMIT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34118) (in query: INSERT INTO test.visits SELECT * FROM datasets.visits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*, String, long&, String, char const*, std::basic_string_view<char, std::char_traits<char>>>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<long&>::type, std::type_identity<String>::type, std::type_identity<char const*>::type, std::type_identity<std::basic_string_view<char, std::char_traits<char>>>::type>, char const*&&, char const*&&, String&&, long&, String&&, char const*&&, std::basic_string_view<char, std::char_traits<char>>&&) @ 0x000000001f173490
3. ./build_docker/./src/Common/MemoryTracker.cpp:340: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f170b2b
4. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f16f9ae
5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f16f9ae
6. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f16f9ae
7. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001f05f046
8. ./build_docker/./src/Common/Allocator.cpp:114: Allocator<false, false>::alloc(unsigned long, unsigned long) @ 0x000000001f05a104
9. ./src/Common/PODArray.h:115: void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 0ul, 0ul>::resize<>(unsigned long) @ 0x00000000316eb6df
10. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:49: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eeb27
11. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
12. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
13. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:142: DB::SerializationString::serializeBinaryBulk(DB::IColumn const&, DB::WriteBuffer&, unsigned long, unsigned long) const @ 0x0000000031ce7e63
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031ba0ede
15. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031bc43b5
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeSingleGranule(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&, DB::ISerialization::SerializeBinaryBulkSettings&, DB::Granule const&) @ 0x00000000393f9a92
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f6050
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
19. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::writeWithPermutation(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x0000000039885612
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039964471
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
23. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
24. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
25. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
26. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
27. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
29. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
31. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be

2024.11.04 23:03:19.382567 [ 1475 ] {30b599b5-4c7a-438e-8b65-1f0946d96df5} <Error> TCPHandler: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 15.49 GiB (attempt to allocate chunk of 2097152 bytes), maximum: 15.30 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*, String, long&, String, char const*, std::basic_string_view<char, std::char_traits<char>>>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<long&>::type, std::type_identity<String>::type, std::type_identity<char const*>::type, std::type_identity<std::basic_string_view<char, std::char_traits<char>>>::type>, char const*&&, char const*&&, String&&, long&, String&&, char const*&&, std::basic_string_view<char, std::char_traits<char>>&&) @ 0x000000001f173490
3. ./build_docker/./src/Common/MemoryTracker.cpp:340: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f170b2b
4. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f16f9ae
5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f16f9ae
6. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001f16f9ae
7. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001f05f046
8. ./build_docker/./src/Common/Allocator.cpp:114: Allocator<false, false>::alloc(unsigned long, unsigned long) @ 0x000000001f05a104
9. ./src/Common/PODArray.h:115: void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 0ul, 0ul>::resize<>(unsigned long) @ 0x00000000316eb6df
10. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:49: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eeb27
11. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
12. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
13. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:142: DB::SerializationString::serializeBinaryBulk(DB::IColumn const&, DB::WriteBuffer&, unsigned long, unsigned long) const @ 0x0000000031ce7e63
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031ba0ede
15. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031bc43b5
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeSingleGranule(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&, DB::ISerialization::SerializeBinaryBulkSettings&, DB::Granule const&) @ 0x00000000393f9a92
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f6050
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
19. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::writeWithPermutation(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x0000000039885612
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039964471
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
23. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
24. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
25. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
26. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
27. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
29. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
31. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be

2024.11.04 23:03:29.161439 [ 2155 ] {} <Error> RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled
2024.11.04 23:03:31.602189 [ 2930 ] {} <Warning> Context: ThreadFuzzer is enabled. Application will run slowly and unstable.
2024.11.04 23:03:31.602285 [ 2930 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.11.04 23:03:31.618758 [ 2930 ] {} <Warning> Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production.
2024.11.04 23:03:31.622518 [ 2930 ] {} <Warning> Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl.
2024.11.04 23:03:31.623893 [ 2930 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.11.04 23:03:55.714650 [ 2930 ] {} <Warning> CloudPlacementInfo: Placement info has not been loaded
2024.11.04 23:03:56.084003 [ 2930 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.11.04 23:03:56.087456 [ 2930 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.11.04 23:03:56.087504 [ 2930 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.11.04 23:03:56.138350 [ 3669 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.11.04 23:03:58.433485 [ 2930 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.11.04 23:04:49.292040 [ 3795 ] {} <Error> ZooKeeperClient: Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<unsigned long&, String>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<String>::type>, unsigned long&, String&&) @ 0x0000000008f26d79
3. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x0000000008f269cd
4. ./src/Common/transformEndianness.h:18: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c1569eb
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
9. ? @ 0x00007fe94d0adac3
10. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:51.336830 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:53.381143 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:53.685786 [ 3816 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:35742 -> [::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: Coordination::ZooKeeper::flushWriteBuffer() @ 0x000000003c13cefe
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:709: Coordination::ZooKeeper::sendThread() @ 0x000000003c153713
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b252
7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
9. ? @ 0x00007fe94d0adac3
10. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:54.210404 [ 2935 ] {2a6488f3-ef04-497d-88ec-a0f5478dcb9b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53770) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:04:54.714424 [ 3885 ] {34dbd2b6-a1f8-405a-80be-f52c1c45b898} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53784) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:55.614097 [ 3894 ] {5fb6998d-ca06-458b-b71b-5178eeceac9c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53840) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:04:55.614169 [ 3893 ] {f6513317-112c-475c-a2d5-4f8b8851cf78} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53902) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:55.614187 [ 3902 ] {84ea8b30-a5c0-4a48-b981-c80f9cc0ac84} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53914) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:55.614275 [ 3927 ] {582aa6e3-aa14-46c2-9b28-a5659c9e94e8} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53928) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:56.111130 [ 3795 ] {} <Error> ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:0: Poco::IOException::IOException(String const&, int) @ 0x000000003e9ff82e
2. ./build_docker/./base/poco/Net/src/NetException.cpp:0: Poco::Net::NetException::NetException(String const&, int) @ 0x000000003e668a8e
3. ./build_docker/./base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x000000003e685f7a
4. ./contrib/llvm-project/libcxx/include/string:1499: Poco::Net::SocketImpl::shutdown() @ 0x000000003e68aeb9
5. ./build_docker/./base/poco/Net/src/StreamSocket.cpp:109: Poco::Net::StreamSocket::shutdown() @ 0x000000003e695dee
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1040: Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000003c13ebb8
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c1563d1
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:56.111271 [ 2935 ] {2a6488f3-ef04-497d-88ec-a0f5478dcb9b} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:56.111378 [ 3949 ] {c0402478-5599-4259-943b-d7479fa4e09d} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46972) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:56.111453 [ 3948 ] {82524e1b-1eb5-49c3-b2ad-5f1f2423a70c} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47030) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:56.111434 [ 3885 ] {34dbd2b6-a1f8-405a-80be-f52c1c45b898} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:56.111402 [ 3936 ] {cf9b443a-34cd-4dd4-b764-96469941b379} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47020) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:56.443100 [ 2933 ] {8df53b9d-a311-406f-9c27-7a34cbb37b4e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58078) (comment: 00014_filtering_arrays.sql) (in query: SELECT GeneralInterests FROM test.hits WHERE AdvEngineID != 0 ORDER BY GeneralInterests DESC LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:04:56.443220 [ 3894 ] {5fb6998d-ca06-458b-b71b-5178eeceac9c} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:56.443244 [ 3893 ] {f6513317-112c-475c-a2d5-4f8b8851cf78} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:56.443224 [ 3999 ] {474b265b-5e84-4d89-80d4-75be3e91a2b8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47034) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:04:56.443374 [ 3927 ] {582aa6e3-aa14-46c2-9b28-a5659c9e94e8} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:56.443319 [ 3902 ] {84ea8b30-a5c0-4a48-b981-c80f9cc0ac84} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:56.759072 [ 4007 ] {3d72d6c6-1e88-4345-ae7a-471862e3d402} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47038) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:04:59.514435 [ 3948 ] {82524e1b-1eb5-49c3-b2ad-5f1f2423a70c} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.514482 [ 3936 ] {cf9b443a-34cd-4dd4-b764-96469941b379} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.514477 [ 3935 ] {91d3ca18-7785-4ff5-a49c-9ee96fd20bf4} <Error> executeQuery: Code: 999. Coordination::Exception: Coordination error: Session expired, path /test/db/test_2. (KEEPER_EXCEPTION) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46940) (comment: 02001_compress_output_file.sh) (in query: CREATE DATABASE IF NOT EXISTS test_a6e4m4lf ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:352: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031e9c650
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
8. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
14. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:04:59.514575 [ 3949 ] {c0402478-5599-4259-943b-d7479fa4e09d} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.514754 [ 2933 ] {8df53b9d-a311-406f-9c27-7a34cbb37b4e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:04:59.514793 [ 4007 ] {3d72d6c6-1e88-4345-ae7a-471862e3d402} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.514796 [ 3999 ] {474b265b-5e84-4d89-80d4-75be3e91a2b8} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.515608 [ 3885 ] {228fb927-6137-46cc-b0db-9496d8b84492} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47134) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:04:59.516724 [ 2935 ] {87353d64-45d0-4331-9d5e-ef5f34ca4409} <Error> executeQuery: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47176) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:04:59.665031 [ 3893 ] {3e15f114-9710-4a59-804e-7a069c4b7409} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58082) (comment: 00384_column_aggregate_function_insert_from.sql) (in query: INSERT INTO aggregates SELECT toDate('2016-10-31') AS d, uniqState(toUInt64(arrayJoin(range(100)))) AS s UNION ALL SELECT toDate('2016-11-01') AS d, uniqState(toUInt64(arrayJoin(range(100)))) AS s;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:04:59.665153 [ 3885 ] {228fb927-6137-46cc-b0db-9496d8b84492} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.665168 [ 3935 ] {91d3ca18-7785-4ff5-a49c-9ee96fd20bf4} <Error> DynamicQueryHandler: Code: 999. Coordination::Exception: Coordination error: Session expired, path /test/db/test_2. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:352: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031e9c650
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
8. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
14. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.665304 [ 2935 ] {87353d64-45d0-4331-9d5e-ef5f34ca4409} <Error> DynamicQueryHandler: Code: 704. DB::Exception: The query result was not cached because the query contains a non-deterministic function. Use setting `query_cache_nondeterministic_function_handling = 'save'` or `= 'ignore'` to cache the query result regardless or to omit caching. (QUERY_CACHE_USED_WITH_NONDETERMINISTIC_FUNCTIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/executeQuery.cpp:1205: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000365a646e
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
6. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:04:59.673840 [ 3893 ] {3e15f114-9710-4a59-804e-7a069c4b7409} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:01.895503 [ 2933 ] {e3efa662-5dc6-476f-858f-a2fe1ed3e9d8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58102) (comment: 01231_operator_null_in.sql) (in query: INSERT INTO null_in VALUES ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:05:03.146796 [ 2933 ] {e3efa662-5dc6-476f-858f-a2fe1ed3e9d8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:05:03.146819 [ 2935 ] {e27cbd48-8c76-4173-b71f-e1f16030e6ef} <Error> executeQuery: Code: 164. DB::Exception: default: Cannot execute query in readonly mode. (READONLY) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58116) (comment: 02315_readonly_create_function.sh) (in query: CREATE FUNCTION test_function AS (x) -> x + 1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Access/ContextAccess.cpp:546: auto bool DB::ContextAccess::checkAccessImplHelper<true, false>(DB::AccessFlags) const::'lambda'<typename ...$T>(int, FormatStringHelperImpl<String, std::type_identity<$T>::type...>, $T&&...)::operator()<>(int, FormatStringHelperImpl<String, std::type_identity<$T>::type...>, $T&&...) const @ 0x0000000030829ef7
4. ./build_docker/./src/Access/ContextAccess.cpp:0: bool DB::ContextAccess::checkAccessImplHelper<true, false>(DB::AccessFlags) const @ 0x0000000030825c94
5. ./build_docker/./src/Access/ContextAccess.cpp:0: bool DB::ContextAccess::checkAccessImplHelper<true, false>(DB::AccessRightsElement const&) const @ 0x000000003084c28b
6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:0: DB::ContextAccess::checkAccess(DB::AccessRightsElements const&) const @ 0x00000000307b5359
7. ./build_docker/./src/Interpreters/Context.cpp:1398: DB::Context::checkAccess(DB::AccessRightsElements const&) const @ 0x00000000328f8b72
8. ./build_docker/./src/Interpreters/InterpreterCreateFunctionQuery.cpp:0: DB::InterpreterCreateFunctionQuery::execute() @ 0x000000003676fd3c
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:05:03.370180 [ 2935 ] {e27cbd48-8c76-4173-b71f-e1f16030e6ef} <Error> TCPHandler: Code: 164. DB::Exception: default: Cannot execute query in readonly mode. (READONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Access/ContextAccess.cpp:546: auto bool DB::ContextAccess::checkAccessImplHelper<true, false>(DB::AccessFlags) const::'lambda'<typename ...$T>(int, FormatStringHelperImpl<String, std::type_identity<$T>::type...>, $T&&...)::operator()<>(int, FormatStringHelperImpl<String, std::type_identity<$T>::type...>, $T&&...) const @ 0x0000000030829ef7
4. ./build_docker/./src/Access/ContextAccess.cpp:0: bool DB::ContextAccess::checkAccessImplHelper<true, false>(DB::AccessFlags) const @ 0x0000000030825c94
5. ./build_docker/./src/Access/ContextAccess.cpp:0: bool DB::ContextAccess::checkAccessImplHelper<true, false>(DB::AccessRightsElement const&) const @ 0x000000003084c28b
6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:0: DB::ContextAccess::checkAccess(DB::AccessRightsElements const&) const @ 0x00000000307b5359
7. ./build_docker/./src/Interpreters/Context.cpp:1398: DB::Context::checkAccess(DB::AccessRightsElements const&) const @ 0x00000000328f8b72
8. ./build_docker/./src/Interpreters/InterpreterCreateFunctionQuery.cpp:0: DB::InterpreterCreateFunctionQuery::execute() @ 0x000000003676fd3c
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:05:04.960821 [ 3893 ] {83b41fdd-3968-4deb-a90c-f18374381afe} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58094) (comment: 00072_compare_date_and_string_index.sql) (in query: SELECT count() FROM test.hits WHERE EventDate = toDate('2014-03-18');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:05:07.185802 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:07.185874 [ 3893 ] {83b41fdd-3968-4deb-a90c-f18374381afe} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:05:07.827208 [ 2935 ] {f2729f83-902f-4561-95e4-5735b1e14574} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions 2 AS value and 1 AS value for alias value. In scope SELECT 1 AS value, 2 AS value. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: SELECT 1 AS value, 2 AS value;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:09.629431 [ 2935 ] {f2729f83-902f-4561-95e4-5735b1e14574} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions 2 AS value and 1 AS value for alias value. In scope SELECT 1 AS value, 2 AS value. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:09.629436 [ 3885 ] {50d26454-1eb0-437d-adee-86e1b725340e} <Error> virtual void DB::StorageBuffer::flushAndPrepareForShutdown(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x00000000387d34a8
9. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAndPrepareForShutdown() @ 0x00000000387d2e4f
10. ./src/Storages/IStorage.h:580: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr<DB::Context const>, DB::ASTDropQuery&, std::shared_ptr<DB::IDatabase>&, StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag>&) @ 0x00000000346b8b46
11. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000346b39bc
12. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:73: DB::InterpreterDropQuery::execute() @ 0x00000000346b0cfe
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
14. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000032b3014e
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr<zkutil::ZooKeeper> const&) @ 0x0000000032b29478
17. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:293: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr<DB::Context const>) @ 0x0000000031f49f20
18. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::QueryFlags) @ 0x0000000031ec1d24
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr<DB::Context const>, DB::ASTDropQuery&, std::shared_ptr<DB::IDatabase>&, StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag>&) @ 0x00000000346b846b
20. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000346b39bc
21. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:73: DB::InterpreterDropQuery::execute() @ 0x00000000346b0cfe
22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:10.496600 [ 3894 ] {cb2e0002-423d-443d-949c-7f19d4574529} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=32): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33984) (comment: 00154_avro.sql) (in query: INSERT INTO test.avro SELECT * FROM test.hits LIMIT 10000;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:05:11.263652 [ 3885 ] {50d26454-1eb0-437d-adee-86e1b725340e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=780, jobs=779). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33980) (comment: 00158_buffer_and_nonexistent_table.sql) (in query: DROP TABLE IF EXISTS test_8l7bivyz.mt_buffer_00158), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:05:11.263745 [ 2935 ] {cbc2aa35-75ad-417f-b4b4-b2090d60c3a7} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions 2 AS value and _CAST(2, 'UInt16') for alias value. In scope SELECT 1 + 1 AS value, 2 AS value. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: SELECT plus(1, 1) AS value, 2 AS value;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:11.263766 [ 3894 ] {cb2e0002-423d-443d-949c-7f19d4574529} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=32): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:05:11.264071 [ 3885 ] {50d26454-1eb0-437d-adee-86e1b725340e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=780, jobs=779). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:05:11.268462 [ 2935 ] {cbc2aa35-75ad-417f-b4b4-b2090d60c3a7} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions 2 AS value and _CAST(2, 'UInt16') for alias value. In scope SELECT 1 + 1 AS value, 2 AS value. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:11.340407 [ 2935 ] {e8b03e80-cec2-45d0-b8ea-da149d92f5fd} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions 1 AS subquery and _CAST(1, 'Nullable(UInt8)') for alias subquery. In scope SELECT (SELECT 1) AS subquery, 1 AS subquery. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: SELECT (SELECT 1) AS subquery, 1 AS subquery;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:11.341077 [ 2935 ] {e8b03e80-cec2-45d0-b8ea-da149d92f5fd} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions 1 AS subquery and _CAST(1, 'Nullable(UInt8)') for alias subquery. In scope SELECT (SELECT 1) AS subquery, 1 AS subquery. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:12.137001 [ 2935 ] {9d14af02-397e-4205-a8ee-6ca119c18265} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions x -> (x + 2) AS lambda and x -> (x + 1) AS lambda for alias lambda. In scope WITH x -> (x + 1) AS lambda, (x -> (x + 2)) AS lambda SELECT lambda(1). (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: WITH x -> x + 1 AS lambda, x -> x + 2 AS lambda SELECT lambda(1);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8309: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a38e3
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:12.913215 [ 2935 ] {9d14af02-397e-4205-a8ee-6ca119c18265} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions x -> (x + 2) AS lambda and x -> (x + 1) AS lambda for alias lambda. In scope WITH x -> (x + 1) AS lambda, (x -> (x + 2)) AS lambda SELECT lambda(1). (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8309: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a38e3
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:12.913272 [ 3894 ] {4dedde18-5eb4-4b7b-96e1-3c549f7f9135} <Error> executeQuery: Code: 352. DB::Exception: Block structure mismatch in (columns with identical name must have identical structure) stream: different types:
1 UInt8 Const(size = 0, UInt8(size = 1))
1 Nullable(Nothing) Const(size = 0, Nullable(size = 1, Nothing(size = 1), UInt8(size = 1))). (AMBIGUOUS_COLUMN_NAME) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34002) (comment: 01950_aliases_bad_cast.sql) (in query: SELECT 1, * FROM (SELECT NULL AS `1`);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<std::basic_string_view<char, std::char_traits<char>>&, String, String>(int, FormatStringHelperImpl<std::type_identity<std::basic_string_view<char, std::char_traits<char>>&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, std::basic_string_view<char, std::char_traits<char>>&, String&&, String&&) @ 0x00000000310335c0
3. ./build_docker/./src/Core/Block.cpp:38: void DB::checkColumnStructure<void>(DB::ColumnWithTypeAndName const&, DB::ColumnWithTypeAndName const&, std::basic_string_view<char, std::char_traits<char>>, bool, int) @ 0x000000003100c40e
4. ./build_docker/./src/Core/Block.cpp:0: DB::Block::insert(DB::ColumnWithTypeAndName) @ 0x000000003100e4cd
5. ./build_docker/./src/Interpreters/ActionsDAG.cpp:0: DB::ActionsDAG::updateHeader(DB::Block) const @ 0x00000000326a9ae0
6. ./src/Core/Block.h:35: DB::ExpressionTransform::transformHeader(DB::Block, DB::ActionsDAG const&) @ 0x000000003affe483
7. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, std::shared_ptr<DB::ActionsDAG> const&) @ 0x000000003b5b5b90
8. ./build_docker/./src/Planner/Planner.cpp:336: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr<DB::ActionsDAG> const&, String const&, std::vector<std::shared_ptr<DB::ActionsDAG>, std::allocator<std::shared_ptr<DB::ActionsDAG>>>&) @ 0x0000000035764a28
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000035757d8d
10. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
11. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:05:14.999377 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:14.999449 [ 3894 ] {4dedde18-5eb4-4b7b-96e1-3c549f7f9135} <Error> TCPHandler: Code: 352. DB::Exception: Block structure mismatch in (columns with identical name must have identical structure) stream: different types:
1 UInt8 Const(size = 0, UInt8(size = 1))
1 Nullable(Nothing) Const(size = 0, Nullable(size = 1, Nothing(size = 1), UInt8(size = 1))). (AMBIGUOUS_COLUMN_NAME), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<std::basic_string_view<char, std::char_traits<char>>&, String, String>(int, FormatStringHelperImpl<std::type_identity<std::basic_string_view<char, std::char_traits<char>>&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, std::basic_string_view<char, std::char_traits<char>>&, String&&, String&&) @ 0x00000000310335c0
3. ./build_docker/./src/Core/Block.cpp:38: void DB::checkColumnStructure<void>(DB::ColumnWithTypeAndName const&, DB::ColumnWithTypeAndName const&, std::basic_string_view<char, std::char_traits<char>>, bool, int) @ 0x000000003100c40e
4. ./build_docker/./src/Core/Block.cpp:0: DB::Block::insert(DB::ColumnWithTypeAndName) @ 0x000000003100e4cd
5. ./build_docker/./src/Interpreters/ActionsDAG.cpp:0: DB::ActionsDAG::updateHeader(DB::Block) const @ 0x00000000326a9ae0
6. ./src/Core/Block.h:35: DB::ExpressionTransform::transformHeader(DB::Block, DB::ActionsDAG const&) @ 0x000000003affe483
7. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, std::shared_ptr<DB::ActionsDAG> const&) @ 0x000000003b5b5b90
8. ./build_docker/./src/Planner/Planner.cpp:336: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr<DB::ActionsDAG> const&, String const&, std::vector<std::shared_ptr<DB::ActionsDAG>, std::allocator<std::shared_ptr<DB::ActionsDAG>>>&) @ 0x0000000035764a28
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000035757d8d
10. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
11. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:05:14.999823 [ 3885 ] {a0ebaac7-cd77-4e4b-a11b-21d8081e23c3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34018) (comment: 00164_quantileBfloat16.sql) (in query: SELECT CounterID AS k, quantileBFloat16(0.5)(ResolutionWidth) FROM test.hits GROUP BY k ORDER BY count() DESC, CounterID LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.015493 [ 3894 ] {8af433b4-d699-4187-bb9a-3ada794b9f32} <Error> executeQuery: Code: 352. DB::Exception: Block structure mismatch in (columns with identical name must have identical structure) stream: different types:
'xyz' String Const(size = 0, String(size = 1))
'xyz' Nullable(Nothing) Const(size = 0, Nullable(size = 1, Nothing(size = 1), UInt8(size = 1))). (AMBIGUOUS_COLUMN_NAME) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34002) (comment: 01950_aliases_bad_cast.sql) (in query: SELECT '7', 'xyz', * FROM (SELECT NULL AS `'xyz'`);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<std::basic_string_view<char, std::char_traits<char>>&, String, String>(int, FormatStringHelperImpl<std::type_identity<std::basic_string_view<char, std::char_traits<char>>&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, std::basic_string_view<char, std::char_traits<char>>&, String&&, String&&) @ 0x00000000310335c0
3. ./build_docker/./src/Core/Block.cpp:38: void DB::checkColumnStructure<void>(DB::ColumnWithTypeAndName const&, DB::ColumnWithTypeAndName const&, std::basic_string_view<char, std::char_traits<char>>, bool, int) @ 0x000000003100c40e
4. ./build_docker/./src/Core/Block.cpp:0: DB::Block::insert(DB::ColumnWithTypeAndName) @ 0x000000003100e4cd
5. ./build_docker/./src/Interpreters/ActionsDAG.cpp:0: DB::ActionsDAG::updateHeader(DB::Block) const @ 0x00000000326a9ae0
6. ./src/Core/Block.h:35: DB::ExpressionTransform::transformHeader(DB::Block, DB::ActionsDAG const&) @ 0x000000003affe483
7. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, std::shared_ptr<DB::ActionsDAG> const&) @ 0x000000003b5b5b90
8. ./build_docker/./src/Planner/Planner.cpp:336: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr<DB::ActionsDAG> const&, String const&, std::vector<std::shared_ptr<DB::ActionsDAG>, std::allocator<std::shared_ptr<DB::ActionsDAG>>>&) @ 0x0000000035764a28
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000035757d8d
10. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
11. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.016090 [ 3894 ] {8af433b4-d699-4187-bb9a-3ada794b9f32} <Error> TCPHandler: Code: 352. DB::Exception: Block structure mismatch in (columns with identical name must have identical structure) stream: different types:
'xyz' String Const(size = 0, String(size = 1))
'xyz' Nullable(Nothing) Const(size = 0, Nullable(size = 1, Nothing(size = 1), UInt8(size = 1))). (AMBIGUOUS_COLUMN_NAME), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<std::basic_string_view<char, std::char_traits<char>>&, String, String>(int, FormatStringHelperImpl<std::type_identity<std::basic_string_view<char, std::char_traits<char>>&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, std::basic_string_view<char, std::char_traits<char>>&, String&&, String&&) @ 0x00000000310335c0
3. ./build_docker/./src/Core/Block.cpp:38: void DB::checkColumnStructure<void>(DB::ColumnWithTypeAndName const&, DB::ColumnWithTypeAndName const&, std::basic_string_view<char, std::char_traits<char>>, bool, int) @ 0x000000003100c40e
4. ./build_docker/./src/Core/Block.cpp:0: DB::Block::insert(DB::ColumnWithTypeAndName) @ 0x000000003100e4cd
5. ./build_docker/./src/Interpreters/ActionsDAG.cpp:0: DB::ActionsDAG::updateHeader(DB::Block) const @ 0x00000000326a9ae0
6. ./src/Core/Block.h:35: DB::ExpressionTransform::transformHeader(DB::Block, DB::ActionsDAG const&) @ 0x000000003affe483
7. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, std::shared_ptr<DB::ActionsDAG> const&) @ 0x000000003b5b5b90
8. ./build_docker/./src/Planner/Planner.cpp:336: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr<DB::ActionsDAG> const&, String const&, std::vector<std::shared_ptr<DB::ActionsDAG>, std::allocator<std::shared_ptr<DB::ActionsDAG>>>&) @ 0x0000000035764a28
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000035757d8d
10. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
11. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.018962 [ 2935 ] {3263cfa7-e246-4b5c-9e33-71cba8c1cc0a} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions _CAST(1, 'Nullable(UInt8)') and (SELECT 1) AS lambda for alias lambda. In scope WITH x -> (x + 1) AS lambda SELECT (SELECT 1) AS lambda. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: WITH x -> x + 1 AS lambda SELECT (SELECT 1) AS lambda;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.019760 [ 2935 ] {3263cfa7-e246-4b5c-9e33-71cba8c1cc0a} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions _CAST(1, 'Nullable(UInt8)') and (SELECT 1) AS lambda for alias lambda. In scope WITH x -> (x + 1) AS lambda SELECT (SELECT 1) AS lambda. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.030632 [ 2935 ] {a79dafbf-3487-403b-8197-5b55891c8716} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions 1 AS lambda and x -> (x + 1) AS lambda for alias lambda. In scope WITH x -> (x + 1) AS lambda SELECT 1 AS lambda. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: WITH x -> x + 1 AS lambda SELECT 1 AS lambda;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8309: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a38e3
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.031310 [ 2935 ] {a79dafbf-3487-403b-8197-5b55891c8716} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions 1 AS lambda and x -> (x + 1) AS lambda for alias lambda. In scope WITH x -> (x + 1) AS lambda SELECT 1 AS lambda. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8309: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a38e3
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.041213 [ 2935 ] {c000bed8-65bd-41d7-87b4-fee257c1e38a} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions test_13.test_table.value and test_13.test_table.id for alias value. In scope SELECT id AS value, value AS value FROM test_table. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: SELECT id AS value, value AS value FROM test_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.041793 [ 2935 ] {c000bed8-65bd-41d7-87b4-fee257c1e38a} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions test_13.test_table.value and test_13.test_table.id for alias value. In scope SELECT id AS value, value AS value FROM test_table. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.051226 [ 2935 ] {081354d5-e1d1-4887-b632-9a9a8cccc6f9} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions test_13.test_table.value and test_13.test_table.id for alias value_1. In scope SELECT id AS value_1, value AS value_1 FROM test_table. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: SELECT id AS value_1, value AS value_1 FROM test_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.051796 [ 2935 ] {081354d5-e1d1-4887-b632-9a9a8cccc6f9} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions test_13.test_table.value and test_13.test_table.id for alias value_1. In scope SELECT id AS value_1, value AS value_1 FROM test_table. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.062824 [ 2935 ] {6154a84e-cfd1-49c2-b53d-b0fb8c041f7b} <Error> executeQuery: Code: 179. DB::Exception: Multiple expressions id + 1 AS value and test_13.test_table.id for alias value. In scope SELECT id AS value, id + 1 AS value FROM test_table. (MULTIPLE_EXPRESSIONS_FOR_ALIAS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33962) (comment: 02344_analyzer_multiple_aliases_for_expression.sql) (in query: SELECT id AS value, (id + 1) AS value FROM test_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.063543 [ 2935 ] {6154a84e-cfd1-49c2-b53d-b0fb8c041f7b} <Error> TCPHandler: Code: 179. DB::Exception: Multiple expressions id + 1 AS value and test_13.test_table.id for alias value. In scope SELECT id AS value, id + 1 AS value FROM test_table. (MULTIPLE_EXPRESSIONS_FOR_ALIAS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String, String&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String>::type>, String&&, String&&, String&, String&&) @ 0x00000000348bd23f
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:8295: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a3bf4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.172989 [ 3885 ] {a0ebaac7-cd77-4e4b-a11b-21d8081e23c3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:15.848089 [ 3885 ] {2235991e-22c1-4e76-b6db-0eaf27d170a5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51166) (comment: 00024_unused_array_join_in_subquery.sql) (in query: SELECT count() FROM (SELECT 1, arrayJoin([1,2,3])) ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:16.012657 [ 3885 ] {2235991e-22c1-4e76-b6db-0eaf27d170a5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:17.858598 [ 3885 ] {2479be28-314c-473a-acfa-edd21e7aa242} <Error> executeQuery: Code: 72. DB::Exception: Cannot parse exponent while reading decimal: Cannot parse Decimal(9, 1) from String: In scope SELECT toDecimal32('e', 1). (CANNOT_PARSE_NUMBER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51172) (comment: 01070_to_decimal_or_null_exception.sql) (in query: SELECT toDecimal32('e', 1);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./src/IO/readDecimalText.h:0: bool DB::readDigits<true, DB::Decimal<int>>(DB::ReadBuffer&, DB::Decimal<int>&, unsigned int&, int&, bool) @ 0x000000003106141a
4. ./src/IO/readDecimalText.h:0: void DB::readDecimalText<DB::Decimal<int>, void>(DB::ReadBuffer&, DB::Decimal<int>&, unsigned int, unsigned int&, bool) @ 0x0000000031a7195b
5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:48: DB::SerializationDecimal<DB::Decimal<int>>::readText(DB::Decimal<int>&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000031bf737e
6. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::(anonymous namespace)::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, (DB::(anonymous namespace)::ConvertFromStringExceptionMode)0, (DB::(anonymous namespace)::ConvertFromStringParsingMode)0>::execute<unsigned int>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, unsigned int) @ 0x0000000008b21e9e
7. bool DB::callOnIndexAndDataType<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x0000000008b10fd2
8. DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, DB::(anonymous namespace)::UnknownMonotonicity>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ad7324
9. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
10. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
12. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
14. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
15. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
17. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
20. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
28. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
29. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
31. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c

2024.11.04 23:05:18.515183 [ 3885 ] {2479be28-314c-473a-acfa-edd21e7aa242} <Error> TCPHandler: Code: 72. DB::Exception: Cannot parse exponent while reading decimal: Cannot parse Decimal(9, 1) from String: In scope SELECT toDecimal32('e', 1). (CANNOT_PARSE_NUMBER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./src/IO/readDecimalText.h:0: bool DB::readDigits<true, DB::Decimal<int>>(DB::ReadBuffer&, DB::Decimal<int>&, unsigned int&, int&, bool) @ 0x000000003106141a
4. ./src/IO/readDecimalText.h:0: void DB::readDecimalText<DB::Decimal<int>, void>(DB::ReadBuffer&, DB::Decimal<int>&, unsigned int, unsigned int&, bool) @ 0x0000000031a7195b
5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:48: DB::SerializationDecimal<DB::Decimal<int>>::readText(DB::Decimal<int>&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000031bf737e
6. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::(anonymous namespace)::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, (DB::(anonymous namespace)::ConvertFromStringExceptionMode)0, (DB::(anonymous namespace)::ConvertFromStringParsingMode)0>::execute<unsigned int>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, unsigned int) @ 0x0000000008b21e9e
7. bool DB::callOnIndexAndDataType<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x0000000008b10fd2
8. DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<int>>, DB::(anonymous namespace)::NameToDecimal32, DB::(anonymous namespace)::UnknownMonotonicity>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ad7324
9. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
10. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
12. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
14. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
15. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
17. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
20. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
28. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
29. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
31. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c

2024.11.04 23:05:18.515241 [ 3902 ] {a04cf8b6-c4c2-4eb3-9176-8d99b5c3ca39} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=0): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51192) (comment: 00159_parallel_formatting_tsv_and_friends.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format TSV), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003b75a78e
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003b73fe83
8. ./contrib/llvm-project/libcxx/include/vector:1562: DB::MergeTreePrefetchedReadPool::PrefetchedReaders::PrefetchedReaders(DB::MergeTreeReadTask::Readers, Priority, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b73e97d
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::MergeTreePrefetchedReadPool::PrefetchedReaders>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreePrefetchedReadPool::PrefetchedReaders, DB::MergeTreeReadTask::Readers, Priority&, DB::MergeTreePrefetchedReadPool&>(DB::MergeTreeReadTask::Readers&&, Priority&, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b75d6fc
10. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:167: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b750566
11. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
12. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
13. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
14. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
15. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
16. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
17. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
19. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
20. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
21. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:05:20.358637 [ 3902 ] {a04cf8b6-c4c2-4eb3-9176-8d99b5c3ca39} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=0): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003b75a78e
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003b73fe83
8. ./contrib/llvm-project/libcxx/include/vector:1562: DB::MergeTreePrefetchedReadPool::PrefetchedReaders::PrefetchedReaders(DB::MergeTreeReadTask::Readers, Priority, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b73e97d
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::MergeTreePrefetchedReadPool::PrefetchedReaders>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreePrefetchedReadPool::PrefetchedReaders, DB::MergeTreeReadTask::Readers, Priority&, DB::MergeTreePrefetchedReadPool&>(DB::MergeTreeReadTask::Readers&&, Priority&, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b75d6fc
10. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:167: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b750566
11. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
12. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
13. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
14. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
15. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
16. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
17. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
19. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
20. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
21. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:05:20.358768 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:21.906945 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:21.907051 [ 3894 ] {0bbdf4f9-1a71-433a-adc8-a1bd321b3c8b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51204) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:21.907073 [ 2933 ] {76a212ee-c902-42e8-851d-e37917b9cfb2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51282) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.835854 [ 2933 ] {76a212ee-c902-42e8-851d-e37917b9cfb2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.835858 [ 3894 ] {0bbdf4f9-1a71-433a-adc8-a1bd321b3c8b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.835827 [ 3936 ] {e72b4f8d-f535-4e3d-86b0-dac33aff2d3d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51266) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.836101 [ 4479 ] {debf876a-4678-4f73-b996-ea79951df065} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51332) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.835424 [ 4509 ] {13fadcfc-072c-4b82-bf39-18abf78e02f3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51468) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.838845 [ 3935 ] {f1d9f499-eff0-491f-a7da-dba398c4faaf} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51220) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.836155 [ 4481 ] {39760f68-6f5c-4d94-8971-aba0a180af98} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51348) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.844109 [ 4479 ] {debf876a-4678-4f73-b996-ea79951df065} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.844128 [ 3885 ] {7eb3d3fe-9070-47f8-97bf-66f2ea20a110} <Error> executeQuery: Code: 72. DB::Exception: Cannot parse exponent while reading decimal: Cannot parse Decimal(18, 2) from String: In scope SELECT toDecimal64('e', 2). (CANNOT_PARSE_NUMBER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51172) (comment: 01070_to_decimal_or_null_exception.sql) (in query: SELECT toDecimal64('e', 2);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./src/IO/readDecimalText.h:0: bool DB::readDigits<true, DB::Decimal<long>>(DB::ReadBuffer&, DB::Decimal<long>&, unsigned int&, int&, bool) @ 0x000000003106295a
4. ./src/IO/readDecimalText.h:0: void DB::readDecimalText<DB::Decimal<long>, void>(DB::ReadBuffer&, DB::Decimal<long>&, unsigned int, unsigned int&, bool) @ 0x0000000031a72b11
5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:48: DB::SerializationDecimal<DB::Decimal<long>>::readText(DB::Decimal<long>&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000031bf9d7e
6. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::(anonymous namespace)::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, (DB::(anonymous namespace)::ConvertFromStringExceptionMode)0, (DB::(anonymous namespace)::ConvertFromStringParsingMode)0>::execute<unsigned int>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, unsigned int) @ 0x0000000008b75df8
7. bool DB::callOnIndexAndDataType<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x0000000008b65093
8. DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, DB::(anonymous namespace)::UnknownMonotonicity>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008b2ad24
9. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
10. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
12. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
14. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
15. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
17. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
20. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
28. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
29. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
31. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c

2024.11.04 23:05:22.844272 [ 3935 ] {f1d9f499-eff0-491f-a7da-dba398c4faaf} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.850132 [ 4481 ] {39760f68-6f5c-4d94-8971-aba0a180af98} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.838853 [ 4514 ] {0dabea15-a23c-4cea-8f71-8690c2e81491} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51480) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.837957 [ 4477 ] {a8441f09-2ce4-4153-b440-c972000f7c2b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51368) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.836134 [ 4484 ] {98d0a614-8eef-4b2c-8131-4f0bae57b5cc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51382) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.853255 [ 4514 ] {0dabea15-a23c-4cea-8f71-8690c2e81491} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.854788 [ 3885 ] {7eb3d3fe-9070-47f8-97bf-66f2ea20a110} <Error> TCPHandler: Code: 72. DB::Exception: Cannot parse exponent while reading decimal: Cannot parse Decimal(18, 2) from String: In scope SELECT toDecimal64('e', 2). (CANNOT_PARSE_NUMBER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./src/IO/readDecimalText.h:0: bool DB::readDigits<true, DB::Decimal<long>>(DB::ReadBuffer&, DB::Decimal<long>&, unsigned int&, int&, bool) @ 0x000000003106295a
4. ./src/IO/readDecimalText.h:0: void DB::readDecimalText<DB::Decimal<long>, void>(DB::ReadBuffer&, DB::Decimal<long>&, unsigned int, unsigned int&, bool) @ 0x0000000031a72b11
5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:48: DB::SerializationDecimal<DB::Decimal<long>>::readText(DB::Decimal<long>&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000031bf9d7e
6. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::(anonymous namespace)::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, (DB::(anonymous namespace)::ConvertFromStringExceptionMode)0, (DB::(anonymous namespace)::ConvertFromStringParsingMode)0>::execute<unsigned int>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, unsigned int) @ 0x0000000008b75df8
7. bool DB::callOnIndexAndDataType<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x0000000008b65093
8. DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<long>>, DB::(anonymous namespace)::NameToDecimal64, DB::(anonymous namespace)::UnknownMonotonicity>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008b2ad24
9. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
10. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
12. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
14. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
15. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
17. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
20. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
28. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
29. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
31. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c

2024.11.04 23:05:22.837975 [ 4483 ] {0634b4ca-65ed-4a1b-8245-4db091343de7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51362) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.856636 [ 4484 ] {98d0a614-8eef-4b2c-8131-4f0bae57b5cc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.837986 [ 4476 ] {9d52d4e4-ad19-472e-a0a6-8893b44a87a1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51322) (comment: 02235_remote_fs_cache_stress.sh) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.838846 [ 4485 ] {5cd2a628-d1c8-45d2-9e65-199980e00747} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51398) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE str = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.900942 [ 3936 ] {e72b4f8d-f535-4e3d-86b0-dac33aff2d3d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.900949 [ 4483 ] {0634b4ca-65ed-4a1b-8245-4db091343de7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.904570 [ 4485 ] {5cd2a628-d1c8-45d2-9e65-199980e00747} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.905789 [ 4476 ] {9d52d4e4-ad19-472e-a0a6-8893b44a87a1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.909945 [ 4505 ] {134143da-50cb-49e2-b2bc-8ce9e6d6e9b8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51454) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE str = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.914770 [ 4505 ] {134143da-50cb-49e2-b2bc-8ce9e6d6e9b8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:22.940499 [ 4509 ] {13fadcfc-072c-4b82-bf39-18abf78e02f3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.126466 [ 4477 ] {a8441f09-2ce4-4153-b440-c972000f7c2b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.126736 [ 3893 ] {877a44e7-d65b-46bb-9c2b-3dcddb650b88} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51288) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.126720 [ 3949 ] {a7ae32ac-9ea1-4adf-939e-20d816754774} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51246) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.126813 [ 3948 ] {ea43fc6e-8d2e-4879-b4b6-44fc347339d0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51230) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE has(arr, 'asdf337');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.127387 [ 3885 ] {fcfaf630-3075-4d82-9ab6-78edcfeadf97} <Error> executeQuery: Code: 72. DB::Exception: Cannot parse exponent while reading decimal: Cannot parse Decimal(38, 3) from String: In scope SELECT toDecimal128('e', 3). (CANNOT_PARSE_NUMBER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51172) (comment: 01070_to_decimal_or_null_exception.sql) (in query: SELECT toDecimal128('e', 3);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./src/IO/readDecimalText.h:0: bool DB::readDigits<true, DB::Decimal<wide::integer<128ul, int>>>(DB::ReadBuffer&, DB::Decimal<wide::integer<128ul, int>>&, unsigned int&, int&, bool) @ 0x0000000031063fa9
4. ./src/IO/readDecimalText.h:0: void DB::readDecimalText<DB::Decimal<wide::integer<128ul, int>>, void>(DB::ReadBuffer&, DB::Decimal<wide::integer<128ul, int>>&, unsigned int, unsigned int&, bool) @ 0x0000000031a73cf3
5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:0: DB::SerializationDecimal<DB::Decimal<wide::integer<128ul, int>>>::readText(DB::Decimal<wide::integer<128ul, int>>&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000031bfcae0
6. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::(anonymous namespace)::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, (DB::(anonymous namespace)::ConvertFromStringExceptionMode)0, (DB::(anonymous namespace)::ConvertFromStringParsingMode)0>::execute<unsigned int>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, unsigned int) @ 0x0000000008bcbdbe
7. bool DB::callOnIndexAndDataType<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x0000000008bbad95
8. DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, DB::(anonymous namespace)::UnknownMonotonicity>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008b7ec64
9. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
10. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
12. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
14. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
15. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
17. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
20. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
28. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
29. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
31. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c

2024.11.04 23:05:24.126576 [ 4521 ] {b247de11-25be-450a-85b2-dfc2a30ce3c5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51494) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE str = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.126720 [ 4482 ] {6d0c1a91-2a74-4820-a4d6-9b651a6a1d98} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51352) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.127301 [ 4487 ] {769a2ceb-b8a5-43be-903e-4cf6098d59ed} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51402) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.127059 [ 4499 ] {7776239f-a58c-4ca1-aaa5-8526e85cbcf6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51438) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.127214 [ 4527 ] {1ca58291-d766-43d7-a5b2-5c76f147b2f8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51516) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.126744 [ 4518 ] {f6dc7774-a80e-4589-88df-1ddfd93dd056} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51488) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE str = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.127308 [ 4473 ] {a13cc4c2-650b-4e5e-ba5d-52b4baaa4996} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51302) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = 'asdf337';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.954890 [ 3948 ] {ea43fc6e-8d2e-4879-b4b6-44fc347339d0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.954917 [ 4482 ] {6d0c1a91-2a74-4820-a4d6-9b651a6a1d98} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.954956 [ 4487 ] {769a2ceb-b8a5-43be-903e-4cf6098d59ed} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.954882 [ 4475 ] {a036c9fe-1a6c-4e00-b75e-b8a4ae9273d7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51310) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE has(arr, 'asdf337');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955072 [ 4518 ] {f6dc7774-a80e-4589-88df-1ddfd93dd056} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.954941 [ 4493 ] {08eb1325-5c0b-47fe-be70-4853c5dd0e58} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51422) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE indexOf(arr, 'asdf337') > 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955122 [ 3885 ] {fcfaf630-3075-4d82-9ab6-78edcfeadf97} <Error> TCPHandler: Code: 72. DB::Exception: Cannot parse exponent while reading decimal: Cannot parse Decimal(38, 3) from String: In scope SELECT toDecimal128('e', 3). (CANNOT_PARSE_NUMBER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./src/IO/readDecimalText.h:0: bool DB::readDigits<true, DB::Decimal<wide::integer<128ul, int>>>(DB::ReadBuffer&, DB::Decimal<wide::integer<128ul, int>>&, unsigned int&, int&, bool) @ 0x0000000031063fa9
4. ./src/IO/readDecimalText.h:0: void DB::readDecimalText<DB::Decimal<wide::integer<128ul, int>>, void>(DB::ReadBuffer&, DB::Decimal<wide::integer<128ul, int>>&, unsigned int, unsigned int&, bool) @ 0x0000000031a73cf3
5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:0: DB::SerializationDecimal<DB::Decimal<wide::integer<128ul, int>>>::readText(DB::Decimal<wide::integer<128ul, int>>&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000031bfcae0
6. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::(anonymous namespace)::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, (DB::(anonymous namespace)::ConvertFromStringExceptionMode)0, (DB::(anonymous namespace)::ConvertFromStringParsingMode)0>::execute<unsigned int>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, unsigned int) @ 0x0000000008bcbdbe
7. bool DB::callOnIndexAndDataType<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, DB::(anonymous namespace)::UnknownMonotonicity>::executeInternal(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x0000000008bbad95
8. DB::(anonymous namespace)::FunctionConvert<DB::DataTypeDecimal<DB::Decimal<wide::integer<128ul, int>>>, DB::(anonymous namespace)::NameToDecimal128, DB::(anonymous namespace)::UnknownMonotonicity>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008b7ec64
9. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
10. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
12. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
14. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
15. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
17. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
20. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
22. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
28. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
29. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
31. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c

2024.11.04 23:05:24.955122 [ 4007 ] {07ed8447-5c69-4196-adda-d0c0d59dcf41} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51258) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE has(arr, 'asdf337');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
16. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
22. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955271 [ 3949 ] {a7ae32ac-9ea1-4adf-939e-20d816754774} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955303 [ 3893 ] {877a44e7-d65b-46bb-9c2b-3dcddb650b88} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955328 [ 4527 ] {1ca58291-d766-43d7-a5b2-5c76f147b2f8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955363 [ 4473 ] {a13cc4c2-650b-4e5e-ba5d-52b4baaa4996} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955160 [ 4521 ] {b247de11-25be-450a-85b2-dfc2a30ce3c5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955501 [ 3999 ] {7b3984a5-190b-4c1d-ba13-d098f1943e08} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51248) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = str;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955511 [ 4480 ] {3f63e321-bf2d-4791-9d90-8bdc6a3ddcc5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51338) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE arr[1] = str;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.955238 [ 4499 ] {7776239f-a58c-4ca1-aaa5-8526e85cbcf6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.959253 [ 4493 ] {08eb1325-5c0b-47fe-be70-4853c5dd0e58} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.976839 [ 4480 ] {3f63e321-bf2d-4791-9d90-8bdc6a3ddcc5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.990650 [ 4475 ] {a036c9fe-1a6c-4e00-b75e-b8a4ae9273d7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:24.996583 [ 4007 ] {07ed8447-5c69-4196-adda-d0c0d59dcf41} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
16. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
22. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:05:25.001828 [ 3999 ] {7b3984a5-190b-4c1d-ba13-d098f1943e08} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:25.054465 [ 3927 ] {81488331-9df8-4ba8-aa65-e438ea7fafea} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51212) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE has(arr, str);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:25.058396 [ 3927 ] {81488331-9df8-4ba8-aa65-e438ea7fafea} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:25.372601 [ 4524 ] {70890fbf-ed62-40a1-b22f-2d9beb781cd6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51508) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411 WHERE has(arr, str);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:25.374740 [ 4524 ] {70890fbf-ed62-40a1-b22f-2d9beb781cd6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:26.050006 [ 4488 ] {cfc0ecb7-426f-44e5-b5bd-3b2dc508853f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51408) (comment: 02235_remote_fs_cache_stress.sh) (in query: SELECT count() FROM t_01411_num WHERE num = 42;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:26.051420 [ 4488 ] {cfc0ecb7-426f-44e5-b5bd-3b2dc508853f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:28.871081 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:31.555261 [ 3935 ] {3ead2149-aceb-45cc-af52-72ce70e8164c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54066) (in query: insert into test select number + 300000, ('lc_str_' || toString(number))::LowCardinality(String) from numbers(100000);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:31.561118 [ 3935 ] {3ead2149-aceb-45cc-af52-72ce70e8164c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:32.144705 [ 2935 ] {f087ce22-16b1-46ec-90f6-c698951f6916} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54094) (in query: select v from test format Null;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:32.146340 [ 2935 ] {f087ce22-16b1-46ec-90f6-c698951f6916} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:36.836928 [ 3935 ] {5b41b8f1-b8b6-40f0-bee6-88211008eb23} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39840) (comment: 00749_inner_join_of_unnamed_subqueries.sql) (in query: INSERT INTO left_table VALUES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:05:37.473319 [ 2935 ] {ec930573-cc14-4509-9266-c7fba3d0085d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39828) (in query: insert into test select number + 100000, number from numbers(100000);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
24. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:05:37.473347 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f35a2ef5-e44d-4e42-94b5-c841517c6ce9::202411_1_6_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:38.825864 [ 3935 ] {5b41b8f1-b8b6-40f0-bee6-88211008eb23} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:05:38.825862 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:40.695080 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:40.695234 [ 2935 ] {ec930573-cc14-4509-9266-c7fba3d0085d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
24. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:05:42.810255 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:44.259469 [ 3757 ] {} <Error> void DB::SystemLog<DB::SessionLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::SessionLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::flushImpl(std::vector<DB::SessionLogElement, std::allocator<DB::SessionLogElement>> const&, unsigned long) @ 0x000000003604be4a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::savingThreadFunction() @ 0x00000000360485a8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f4369c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43689f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:46.191703 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:47.571395 [ 3948 ] {a60682ca-f5ba-4a62-a081-c1d7f7d6ec1f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=2): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39844) (comment: 00159_parallel_formatting_tsv_and_friends.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format TSVWithNames), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b75050e
19. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
20. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:05:49.469735 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:49.469838 [ 3935 ] {2dcde006-1844-4818-988c-6fa2c5da8f9b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58428) (comment: 00424_shard_aggregate_functions_of_nullable.sql) (in query: SELECT avg(arrayJoin([NULL]));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:49.469875 [ 3948 ] {a60682ca-f5ba-4a62-a081-c1d7f7d6ec1f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=2): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b75050e
19. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
20. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:05:50.263481 [ 3935 ] {2dcde006-1844-4818-988c-6fa2c5da8f9b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:50.263571 [ 2935 ] {e93d2ff5-4693-4711-ac90-7ae442d9165b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58442) (in query: select count() from test where isNotNull(v.`Array(UInt64)`.size0);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:50.264855 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_6_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:50.265057 [ 2935 ] {e93d2ff5-4693-4711-ac90-7ae442d9165b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:50.775056 [ 3935 ] {74edeae9-146b-4fe4-9905-3af329a24a51} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58468) (comment: 00159_parallel_formatting_tsv_and_friends.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format TSKV), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:50.779587 [ 3935 ] {74edeae9-146b-4fe4-9905-3af329a24a51} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:51.474229 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_6_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:51.763635 [ 2935 ] {f1e776a2-c5d9-4c68-aa92-5ca1433909c1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58492) (in query: select count() from test where isNotNull(v);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:51.766256 [ 2935 ] {f1e776a2-c5d9-4c68-aa92-5ca1433909c1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:55.899693 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:05:56.638217 [ 2935 ] {204fc989-b0c7-43fe-ab18-80c2d92c44f7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44926) (in query: insert into test select number + 100000, number from numbers(100000);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:176: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b005
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
25. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
26. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
27. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:05:56.642654 [ 2935 ] {204fc989-b0c7-43fe-ab18-80c2d92c44f7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:176: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b005
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
25. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
26. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
27. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:05:57.811553 [ 2935 ] {5dd5cf69-7bc9-48fd-b76a-5943ffc0859b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44934) (in query: select count() from test where isNotNull(v.String);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:05:57.813726 [ 2935 ] {5dd5cf69-7bc9-48fd-b76a-5943ffc0859b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:00.422501 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:01.587554 [ 2935 ] {c9d0ad42-e914-4795-b153-272055b12345} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44952) (in query: select count() from test where isNotNull(v);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationVariant::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031d1de3d
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:06:03.381131 [ 2935 ] {c9d0ad42-e914-4795-b153-272055b12345} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationVariant::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031d1de3d
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:06:03.381155 [ 3935 ] {b2bfe87d-9d55-4217-b118-ab1cf860914c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44962) (comment: 00022_merge_prewhere.sql) (in query: SELECT count() FROM test.merge_hits WHERE AdvEngineID = 2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Storages/StorageMerge.cpp:0: DB::ReadFromMerge::buildPipeline(DB::ReadFromMerge::ChildPlan&, DB::QueryProcessingStage::Enum) const @ 0x0000000037f4b721
15. ./build_docker/./src/Storages/StorageMerge.cpp:451: DB::ReadFromMerge::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000037f47f1b
16. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
17. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
23. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
27. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
28. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:06:03.382769 [ 3935 ] {b2bfe87d-9d55-4217-b118-ab1cf860914c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Storages/StorageMerge.cpp:0: DB::ReadFromMerge::buildPipeline(DB::ReadFromMerge::ChildPlan&, DB::QueryProcessingStage::Enum) const @ 0x0000000037f4b721
15. ./build_docker/./src/Storages/StorageMerge.cpp:451: DB::ReadFromMerge::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000037f47f1b
16. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
17. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
23. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
27. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
28. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:06:06.051401 [ 3935 ] {e4d0acac-c26f-4e70-a0b6-4c9ab96ac5ce} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=30): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59304) (comment: 00157_cache_dictionary.sql) (in query: INSERT INTO test.hits_1m SELECT * FROM test.hits LIMIT 1000000;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
29. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:06.063629 [ 3935 ] {e4d0acac-c26f-4e70-a0b6-4c9ab96ac5ce} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=30): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
29. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:08.144797 [ 2935 ] {dddd1a69-2b1d-49ce-9222-312a99cdb045} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=57, jobs=56). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59330) (comment: 02735_parquet_encoder.sql) (in query: insert into function file(basic_types_02735.parquet) select * from basic_types_02735;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Formats/Impl/ParquetBlockOutputFormat.cpp:0: DB::ParquetBlockOutputFormat::startMoreThreadsIfNeeded(std::unique_lock<std::mutex> const&) @ 0x000000003ac4ba80
7. ./contrib/llvm-project/libcxx/include/vector:434: DB::ParquetBlockOutputFormat::threadFunction() @ 0x000000003ac4e9db
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ParquetBlockOutputFormat::startMoreThreadsIfNeeded(std::unique_lock<std::mutex> const&)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003ac50774
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<bool ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<bool>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(bool&&)::'lambda'()::operator()() @ 0x000000001f35ae33
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35acdf
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:06:08.148417 [ 2935 ] {dddd1a69-2b1d-49ce-9222-312a99cdb045} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=57, jobs=56). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Formats/Impl/ParquetBlockOutputFormat.cpp:0: DB::ParquetBlockOutputFormat::startMoreThreadsIfNeeded(std::unique_lock<std::mutex> const&) @ 0x000000003ac4ba80
7. ./contrib/llvm-project/libcxx/include/vector:434: DB::ParquetBlockOutputFormat::threadFunction() @ 0x000000003ac4e9db
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ParquetBlockOutputFormat::startMoreThreadsIfNeeded(std::unique_lock<std::mutex> const&)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003ac50774
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<bool ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<bool>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(bool&&)::'lambda'()::operator()() @ 0x000000001f35ae33
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35acdf
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:06:09.525580 [ 3935 ] {f84386dc-5143-4ea6-9b91-0ed4f4c29677} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59340) (comment: 00159_parallel_formatting_json_and_friends_1.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONEachRow), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:11.078915 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:11.079052 [ 3935 ] {f84386dc-5143-4ea6-9b91-0ed4f4c29677} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:13.083759 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:13.083959 [ 2935 ] {858c5f31-5f9d-4581-b5f0-3ea875417b63} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59344) (comment: 01271_optimize_arithmetic_operations_in_aggr_func_long.sql) (in query: SELECT sum(n + 1), sum(1 + n), sum(n - 1), sum(1 - n) FROM (SELECT number n FROM numbers(10));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:13.087921 [ 2935 ] {858c5f31-5f9d-4581-b5f0-3ea875417b63} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:14.933725 [ 3935 ] {3c752274-a9c9-4f7f-a3fb-4fd0720103e3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38450) (comment: 01651_bugs_from_15889.sql) (in query: INSERT INTO xp SELECT '2020-01-01', number, '' FROM numbers(100000);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
25. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
26. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
27. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:06:14.937496 [ 3935 ] {3c752274-a9c9-4f7f-a3fb-4fd0720103e3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
25. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
26. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
27. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:06:17.658721 [ 3948 ] {50eee90c-a1d2-4a90-8b5f-0b8c1b65bff0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38454) (comment: 02459_low_cardinality_uint128_aggregator.sql) (in query: INSERT INTO group_by_pk_lc_uint128 SELECT number / 100, number FROM numbers(1000);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
25. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:06:17.660646 [ 3948 ] {50eee90c-a1d2-4a90-8b5f-0b8c1b65bff0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
25. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:06:19.408305 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:21.026995 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:22.994152 [ 4756 ] {} <Error> void DB::AsyncLoader::worker(Pool &): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=805). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000032b5d944
7. ./build_docker/./src/Interpreters/DDLWorker.cpp:131: DB::DDLWorker::startup() @ 0x0000000032b0aaf4
8. ./build_docker/./src/Databases/DatabaseReplicated.cpp:579: void std::__function::__policy_invoker<void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>::__call_impl<std::__function::__default_alloc_func<DB::DatabaseReplicated::startupDatabaseAsync(DB::AsyncLoader&, std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>, DB::LoadingStrictnessLevel)::$_0, void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>>(std::__function::__policy_storage const*, DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&) @ 0x0000000031f0eaa5
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001f739580
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f73bb40
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:23.317715 [ 4764 ] {} <Error> DDLWorker(test_c8ab4ec7): Error on initialization of test_c8ab4ec7: Code: 695. DB::Exception: Load job 'startup Replicated database test_c8ab4ec7' failed: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=805). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000032b5d944
7. ./build_docker/./src/Interpreters/DDLWorker.cpp:131: DB::DDLWorker::startup() @ 0x0000000032b0aaf4
8. ./build_docker/./src/Databases/DatabaseReplicated.cpp:579: void std::__function::__policy_invoker<void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>::__call_impl<std::__function::__default_alloc_func<DB::DatabaseReplicated::startupDatabaseAsync(DB::AsyncLoader&, std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>, DB::LoadingStrictnessLevel)::$_0, void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>>(std::__function::__policy_storage const*, DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&) @ 0x0000000031f0eaa5
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001f739580
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f73bb40
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build)). (ASYNC_LOAD_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x000000000a10afd5
3. ./build_docker/./src/Common/AsyncLoader.cpp:942: DB::AsyncLoader::worker(DB::AsyncLoader::Pool&) @ 0x000000001f73af2c
4. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f73bb40
5. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
6. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:24.038949 [ 3949 ] {cc448275-5d99-4c49-8bb7-f534aeea0e19} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43440) (comment: 00159_parallel_formatting_json_and_friends_1.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactEachRow), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:24.067603 [ 3949 ] {cc448275-5d99-4c49-8bb7-f534aeea0e19} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:24.997925 [ 3949 ] {64cd9d42-2bcf-4c16-a3da-ad1ece64e131} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43442) (comment: 00159_parallel_formatting_json_and_friends_1.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactEachRow), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:25.050708 [ 3949 ] {64cd9d42-2bcf-4c16-a3da-ad1ece64e131} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:25.628865 [ 3949 ] {f38df01e-b010-490d-afdc-a95cd050cb67} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43448) (comment: 00159_parallel_formatting_json_and_friends_1.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactStringsEachRow), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:25.632336 [ 3949 ] {f38df01e-b010-490d-afdc-a95cd050cb67} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:28.322234 [ 3948 ] {4b620648-c536-4f35-9777-4d177dcca743} <Error> executeQuery: Code: 695. DB::Exception: Load job 'startup Replicated database test_c8ab4ec7' failed: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=805). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000032b5d944
7. ./build_docker/./src/Interpreters/DDLWorker.cpp:131: DB::DDLWorker::startup() @ 0x0000000032b0aaf4
8. ./build_docker/./src/Databases/DatabaseReplicated.cpp:579: void std::__function::__policy_invoker<void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>::__call_impl<std::__function::__default_alloc_func<DB::DatabaseReplicated::startupDatabaseAsync(DB::AsyncLoader&, std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>, DB::LoadingStrictnessLevel)::$_0, void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>>(std::__function::__policy_storage const*, DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&) @ 0x0000000031f0eaa5
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001f739580
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f73bb40
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build)). (ASYNC_LOAD_FAILED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45474) (comment: 01143_trivial_count_with_join.sql) (in query: CREATE DATABASE IF NOT EXISTS test_c8ab4ec7 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x000000000a10afd5
3. ./build_docker/./src/Common/AsyncLoader.cpp:942: DB::AsyncLoader::worker(DB::AsyncLoader::Pool&) @ 0x000000001f73af2c
4. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f73bb40
5. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
6. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:06:28.325754 [ 3948 ] {4b620648-c536-4f35-9777-4d177dcca743} <Error> DynamicQueryHandler: Code: 695. DB::Exception: Load job 'startup Replicated database test_c8ab4ec7' failed: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=805). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000032b5d944
7. ./build_docker/./src/Interpreters/DDLWorker.cpp:131: DB::DDLWorker::startup() @ 0x0000000032b0aaf4
8. ./build_docker/./src/Databases/DatabaseReplicated.cpp:579: void std::__function::__policy_invoker<void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>::__call_impl<std::__function::__default_alloc_func<DB::DatabaseReplicated::startupDatabaseAsync(DB::AsyncLoader&, std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>, DB::LoadingStrictnessLevel)::$_0, void (DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&)>>(std::__function::__policy_storage const*, DB::AsyncLoader&, std::shared_ptr<DB::LoadJob> const&) @ 0x0000000031f0eaa5
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001f739580
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f73bb40
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build)). (ASYNC_LOAD_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x000000000a10afd5
3. ./build_docker/./src/Common/AsyncLoader.cpp:942: DB::AsyncLoader::worker(DB::AsyncLoader::Pool&) @ 0x000000001f73af2c
4. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f73bb40
5. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
6. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.044653 [ 3999 ] {92496871-d281-4e1e-a6a8-a579dc2b39c7} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:a8a6da11-8090-41ca-a912-7961391b6907'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36676) (comment: 01921_test_progress_bar.py) (in query: CREATE DATABASE IF NOT EXISTS test_6v2tlntc ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.046040 [ 3999 ] {92496871-d281-4e1e-a6a8-a579dc2b39c7} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:a8a6da11-8090-41ca-a912-7961391b6907'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.059859 [ 3948 ] {abae5777-5e9f-42d1-a6be-15b19a24032c} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:5bd98f4a-d26d-4455-9451-a3172d1a1254'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36692) (comment: 02932_query_settings_max_size_drop.sql) (in query: CREATE DATABASE IF NOT EXISTS test_u5zs1w4g ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.061375 [ 3948 ] {abae5777-5e9f-42d1-a6be-15b19a24032c} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:5bd98f4a-d26d-4455-9451-a3172d1a1254'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.071643 [ 3948 ] {96e90238-5865-4a9f-9fb8-347b954285e2} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:54e0ddba-0741-4438-9886-936910dce6a0'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36696) (comment: 02402_capnp_format_segments_overflow.sh) (in query: CREATE DATABASE IF NOT EXISTS test_27qyr0p5 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.072920 [ 3948 ] {96e90238-5865-4a9f-9fb8-347b954285e2} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:54e0ddba-0741-4438-9886-936910dce6a0'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.095788 [ 3948 ] {19918eae-8058-4895-80fe-f11a3a406a39} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:2f52a3f9-d929-4f2b-aa34-1afd169afae0'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36702) (comment: 01834_alias_columns_laziness_filimonov.sh) (in query: CREATE DATABASE IF NOT EXISTS test_8j6podt0 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.183709 [ 3948 ] {19918eae-8058-4895-80fe-f11a3a406a39} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:2f52a3f9-d929-4f2b-aa34-1afd169afae0'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.196853 [ 3948 ] {17dd5ed7-987e-42e0-9ea9-5e153a07c21b} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:6a5de139-ef4c-4bc0-b51a-80a27686cd23'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36710) (comment: 01536_fuzz_cast.sql) (in query: CREATE DATABASE IF NOT EXISTS test_kshby05j ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.198265 [ 3948 ] {17dd5ed7-987e-42e0-9ea9-5e153a07c21b} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:6a5de139-ef4c-4bc0-b51a-80a27686cd23'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.209875 [ 3948 ] {22166841-ce19-4b88-8158-0519b55b621b} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:657c0cf9-5af8-4ddb-bca9-e50c0970eaaa'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36718) (comment: 01273_h3EdgeAngle_range_check.sql) (in query: CREATE DATABASE IF NOT EXISTS test_kc62113i ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.211086 [ 3948 ] {22166841-ce19-4b88-8158-0519b55b621b} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:657c0cf9-5af8-4ddb-bca9-e50c0970eaaa'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.317193 [ 3948 ] {3477e2ec-5328-4dc9-a399-dd1f4a101660} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:3aa3d4b7-f517-4b35-b1d6-75a8d1ad306f'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36726) (comment: 02250_hints_for_columns.sh) (in query: CREATE DATABASE IF NOT EXISTS test_2le6vhn4 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.318711 [ 3948 ] {3477e2ec-5328-4dc9-a399-dd1f4a101660} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:3aa3d4b7-f517-4b35-b1d6-75a8d1ad306f'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.331063 [ 3948 ] {1903b6a9-1ec9-4cab-acc6-52a77eae8fcd} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:03b48368-6345-40d6-9b51-216ee130d051'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36738) (comment: 02873_s3_presigned_url_and_url_with_special_characters.sql) (in query: CREATE DATABASE IF NOT EXISTS test_uyw55j9z ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.332417 [ 3948 ] {1903b6a9-1ec9-4cab-acc6-52a77eae8fcd} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:03b48368-6345-40d6-9b51-216ee130d051'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.348937 [ 3948 ] {ff9a9466-3e1a-410d-83fa-8edb2552f064} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:80ee532a-6c16-46eb-a486-ba9f2af440ae'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36748) (comment: 01710_aggregate_projection_with_grouping_set.sql) (in query: CREATE DATABASE IF NOT EXISTS test_mo3o861q ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.350201 [ 3948 ] {ff9a9466-3e1a-410d-83fa-8edb2552f064} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:80ee532a-6c16-46eb-a486-ba9f2af440ae'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.363639 [ 3948 ] {eab6db28-e14b-4c59-a8a7-99d187e48cce} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:68c604ae-2cbc-4903-9b6c-69c484cf07cb'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36752) (comment: 02875_clickhouse_local_multiquery.sh) (in query: CREATE DATABASE IF NOT EXISTS test_q49w625w ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.373928 [ 3948 ] {eab6db28-e14b-4c59-a8a7-99d187e48cce} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:68c604ae-2cbc-4903-9b6c-69c484cf07cb'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.386682 [ 3948 ] {d1aece75-1bb2-42d0-8413-5996bead32f3} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:b9bcc47b-1ff5-48d0-a8b4-33b84f0dc623'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36766) (comment: 02676_distinct_reading_in_order_analyzer.sql) (in query: CREATE DATABASE IF NOT EXISTS test_x4ae1f4e ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.388175 [ 3948 ] {d1aece75-1bb2-42d0-8413-5996bead32f3} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:b9bcc47b-1ff5-48d0-a8b4-33b84f0dc623'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.399908 [ 3948 ] {a148b84c-1081-4d79-ac44-6d18b064c17d} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:8f542a96-a25f-4c8b-8594-7081e10978b8'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36768) (comment: 02990_format_select_from_explain.sh) (in query: CREATE DATABASE IF NOT EXISTS test_zcz5habl ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.401425 [ 3948 ] {a148b84c-1081-4d79-ac44-6d18b064c17d} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:8f542a96-a25f-4c8b-8594-7081e10978b8'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.413484 [ 3948 ] {04efdd0b-924a-41b0-aeeb-1d85d9d5c0d8} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:d14f029c-9ac9-4dcd-af88-018f519d3b14'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36776) (comment: 00170_lower_upper_utf8.sql) (in query: CREATE DATABASE IF NOT EXISTS test_k9j3ini5 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.414816 [ 3948 ] {04efdd0b-924a-41b0-aeeb-1d85d9d5c0d8} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:d14f029c-9ac9-4dcd-af88-018f519d3b14'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.450135 [ 4007 ] {e8e9aa9c-bd67-4e0e-a473-d6f7cb2ba83b} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:acefb5fe-955e-40ef-b891-29bb4f23bdea'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36794) (comment: 00014_filtering_arrays.sql) (in query: CREATE DATABASE IF NOT EXISTS test_atpb121n ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.451921 [ 4007 ] {e8e9aa9c-bd67-4e0e-a473-d6f7cb2ba83b} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:acefb5fe-955e-40ef-b891-29bb4f23bdea'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.464105 [ 4007 ] {e0d529d1-fdc5-4443-bfa2-2e9522f63396} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:378b6bf3-8346-46d3-b250-005193270c43'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36800) (comment: 00076_system_columns_bytes.sql) (in query: CREATE DATABASE IF NOT EXISTS test_ccgghadq ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.465563 [ 4007 ] {e0d529d1-fdc5-4443-bfa2-2e9522f63396} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:378b6bf3-8346-46d3-b250-005193270c43'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.479285 [ 4007 ] {8e3e34f9-da19-4dde-907a-eaa84804b82e} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:fe4a797b-3416-4712-9fe5-3d3595b6f5ce'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36802) (comment: 00038_uniq_state_merge2.sql) (in query: CREATE DATABASE IF NOT EXISTS test_rqtzv6ur ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.480913 [ 4007 ] {8e3e34f9-da19-4dde-907a-eaa84804b82e} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:fe4a797b-3416-4712-9fe5-3d3595b6f5ce'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.496089 [ 4007 ] {be204c2e-e35b-4248-93fc-1fdbbe1f9ca1} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:ff9c07c5-e553-4fc8-839a-9aaf49e7868a'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36812) (comment: 00024_random_counters.sql) (in query: CREATE DATABASE IF NOT EXISTS test_aooco2y2 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.497375 [ 4007 ] {be204c2e-e35b-4248-93fc-1fdbbe1f9ca1} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:ff9c07c5-e553-4fc8-839a-9aaf49e7868a'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.509893 [ 4007 ] {3c08d132-a958-4d4e-afcf-fc01b547b232} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:a5ba9a62-a729-4206-883a-a579441c2c11'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36818) (comment: 00045_uniq_upto.sql) (in query: CREATE DATABASE IF NOT EXISTS test_4cavxac2 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.511412 [ 4007 ] {3c08d132-a958-4d4e-afcf-fc01b547b232} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:a5ba9a62-a729-4206-883a-a579441c2c11'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.525485 [ 4007 ] {51afb2b0-623d-4969-8bde-a7139a976be6} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:4c465073-8e39-4b1e-9107-460b17d2ce72'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36824) (comment: 00097_constexpr_in_index.sql) (in query: CREATE DATABASE IF NOT EXISTS test_8d1jy2cq ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.526994 [ 4007 ] {51afb2b0-623d-4969-8bde-a7139a976be6} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:4c465073-8e39-4b1e-9107-460b17d2ce72'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.539438 [ 4007 ] {97cc7515-eed4-4d5f-a4aa-2be21fbd6e8d} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:f47be3ef-2c66-4833-9454-3ef5888553a8'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36828) (comment: 00077_log_tinylog_stripelog.sql) (in query: CREATE DATABASE IF NOT EXISTS test_7rqewpmp ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.541080 [ 4007 ] {97cc7515-eed4-4d5f-a4aa-2be21fbd6e8d} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:f47be3ef-2c66-4833-9454-3ef5888553a8'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.555008 [ 4007 ] {821890f1-927c-4ca3-b290-9f0f4fe8c1a6} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:c82ef828-9091-4978-9239-781373002070'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36844) (comment: 00013_sorting_of_nested.sql) (in query: CREATE DATABASE IF NOT EXISTS test_7bsvbm1g ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.556525 [ 4007 ] {821890f1-927c-4ca3-b290-9f0f4fe8c1a6} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:c82ef828-9091-4978-9239-781373002070'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.569888 [ 4007 ] {48410f48-62a8-4e3d-8348-569d3698a5d9} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:f44bf7b8-37a8-4fb1-a810-5b15eb1672c1'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36860) (comment: 00055_index_and_not.sql) (in query: CREATE DATABASE IF NOT EXISTS test_u9k0lmg4 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.571250 [ 4007 ] {48410f48-62a8-4e3d-8348-569d3698a5d9} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:f44bf7b8-37a8-4fb1-a810-5b15eb1672c1'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.583979 [ 4007 ] {0349c6a8-812a-4ab9-9a29-8e0553809184} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:66b9296f-0e8f-4253-b17a-a80a7055be04'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36868) (comment: 00098_primary_key_memory_allocated.sql) (in query: CREATE DATABASE IF NOT EXISTS test_q5qa61c0 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.585354 [ 4007 ] {0349c6a8-812a-4ab9-9a29-8e0553809184} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:66b9296f-0e8f-4253-b17a-a80a7055be04'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.597877 [ 4007 ] {c0299530-df91-4885-992c-edddc7a26399} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:e5d97ed8-91c3-42bf-b832-532e5092da1b'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36874) (comment: 00164_quantileBfloat16.sql) (in query: CREATE DATABASE IF NOT EXISTS test_d1lvx4r4 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.599048 [ 4007 ] {c0299530-df91-4885-992c-edddc7a26399} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:e5d97ed8-91c3-42bf-b832-532e5092da1b'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.610704 [ 4007 ] {d74cbc32-b2ab-4c1a-9852-e7bd7a0c474b} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:9ee9fde2-fc83-40f7-a349-25827e676796'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36890) (comment: 00160_decode_xml_component.sql) (in query: CREATE DATABASE IF NOT EXISTS test_5kj6wrt7 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.611886 [ 4007 ] {d74cbc32-b2ab-4c1a-9852-e7bd7a0c474b} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:9ee9fde2-fc83-40f7-a349-25827e676796'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.635287 [ 4007 ] {80e8cf0a-7baf-4f4b-8344-92d9c31ec7db} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:7cd17987-01b4-4faf-a890-355d5fb57e9d'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36892) (comment: 00086_array_reduce.sql) (in query: CREATE DATABASE IF NOT EXISTS test_n9dbq7lo ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.636888 [ 4007 ] {80e8cf0a-7baf-4f4b-8344-92d9c31ec7db} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:7cd17987-01b4-4faf-a890-355d5fb57e9d'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.648894 [ 3948 ] {8ed99690-6feb-4d11-8da0-bd67ad3c8793} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:4db6338a-b108-478e-ae49-3d2bbbba49d3'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36908) (comment: 00016_any_if_distributed_cond_always_false.sql) (in query: CREATE DATABASE IF NOT EXISTS test_rbraesdr ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.650203 [ 3948 ] {8ed99690-6feb-4d11-8da0-bd67ad3c8793} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:4db6338a-b108-478e-ae49-3d2bbbba49d3'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.660869 [ 3948 ] {231f0bc9-6764-41f0-a979-e94c1a751c88} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:332068ca-5f04-4706-a858-226305b05b6f'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36924) (comment: 00150_quantiles_timing_precision.sql) (in query: CREATE DATABASE IF NOT EXISTS test_kcav6lh5 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.662006 [ 3948 ] {231f0bc9-6764-41f0-a979-e94c1a751c88} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:332068ca-5f04-4706-a858-226305b05b6f'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.679935 [ 3999 ] {73463190-1828-469c-8382-b4f22c00b2a8} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:a97979d8-5380-4beb-aeea-524e41724196'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36936) (comment: 00172_hits_joins.gen.sql) (in query: CREATE DATABASE IF NOT EXISTS test_6j2ce18n ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.682816 [ 3999 ] {73463190-1828-469c-8382-b4f22c00b2a8} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:a97979d8-5380-4beb-aeea-524e41724196'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.693939 [ 3948 ] {9fce6dc2-62a8-4d5a-b1fd-7afb187bea5a} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:dd4c4923-250f-459b-b75e-ed038b246ac1'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36942) (comment: 00180_no_seek_avoiding_when_reading_from_cache.sh) (in query: CREATE DATABASE IF NOT EXISTS test_kn7fz7ph ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.695110 [ 3948 ] {9fce6dc2-62a8-4d5a-b1fd-7afb187bea5a} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:dd4c4923-250f-459b-b75e-ed038b246ac1'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.704334 [ 3948 ] {31332fcc-0765-45b0-b007-dfb19b884920} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:1a194389-f3a9-47c1-95d5-a12d5802fb64'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36956) (comment: 00020_distinct_order_by_distributed.sql) (in query: CREATE DATABASE IF NOT EXISTS test_81wrbfyw ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.705318 [ 3948 ] {31332fcc-0765-45b0-b007-dfb19b884920} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:1a194389-f3a9-47c1-95d5-a12d5802fb64'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.739637 [ 3948 ] {c65e053e-ed96-447f-9bad-a15689e47964} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:ae0c271f-ea55-4cef-9ac9-d5b1673059b3'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36972) (comment: 00085_monotonic_evaluation_segfault.sql) (in query: CREATE DATABASE IF NOT EXISTS test_tktsx8a6 ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.741152 [ 3948 ] {c65e053e-ed96-447f-9bad-a15689e47964} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:ae0c271f-ea55-4cef-9ac9-d5b1673059b3'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:29.755739 [ 3948 ] {4e749f63-2296-40a6-98dc-60a6167716b0} <Error> executeQuery: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:1e235635-df86-424c-abde-3b7bcd094114'. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36984) (comment: 00015_totals_and_no_aggregate_functions.sql) (in query: CREATE DATABASE IF NOT EXISTS test_l3lpbz2a ENGINE=Replicated('/test/db/test_2', 's1', 'r1') ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:06:29.840975 [ 3948 ] {4e749f63-2296-40a6-98dc-60a6167716b0} <Error> DynamicQueryHandler: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/db/test_2 already exists. Replica host ID: 'd7407d092bf4:9000:51c77011-51df-4f10-8315-e67bc83af69b', current host ID: 'd7407d092bf4:9000:1e235635-df86-424c-abde-3b7bcd094114'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String&, String&, String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&, String&, String&, String&) @ 0x0000000031f14550
3. ./build_docker/./src/Databases/DatabaseReplicated.cpp:375: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000031ea0f43
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr<DB::Context>, DB::LoadingStrictnessLevel) @ 0x0000000031eb2198
5. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set<std::shared_ptr<DB::LoadJob>, std::hash<std::shared_ptr<DB::LoadJob>>, std::equal_to<std::shared_ptr<DB::LoadJob>>, std::allocator<std::shared_ptr<DB::LoadJob>>>) @ 0x0000000034691c51
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x00000000346071bb
7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
11. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:32.647827 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:36.223428 [ 3949 ] {c65152ca-9a91-4740-91d1-32cddafe2ac5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48136) (comment: 00159_parallel_formatting_json_and_friends_1.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactEachRowWithNames), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:06:36.258863 [ 3949 ] {c65152ca-9a91-4740-91d1-32cddafe2ac5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:06:37.147193 [ 3949 ] {19196311-b01c-4b4a-9f90-e13a551b272a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48184) (comment: 00159_parallel_formatting_json_and_friends_1.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactEachRowWithNames), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:37.176928 [ 3949 ] {19196311-b01c-4b4a-9f90-e13a551b272a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:38.191424 [ 3999 ] {9c4e0ff9-df15-408c-a1c2-89cd66dec471} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=786). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48222) (comment: 00013_sorting_of_nested.sql) (in query: SELECT ParsedParams.Key1 FROM test.visits FINAL WHERE VisitID != 0 AND notEmpty(ParsedParams.Key1) ORDER BY VisitID LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:06:38.191476 [ 3949 ] {4da18300-db7d-421a-aed1-300d0c301267} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48210) (comment: 02555_davengers_rename_chain.sh) (in query: SELECT * FROM wrong_metadata order by a FORMAT JSONEachRow), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:38.193664 [ 3999 ] {9c4e0ff9-df15-408c-a1c2-89cd66dec471} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=786). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:06:38.197718 [ 3949 ] {4da18300-db7d-421a-aed1-300d0c301267} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:06:40.658907 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:40.659043 [ 3999 ] {4fb413f5-eb75-45bc-bd53-43dd06616ba2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48236) (comment: 00165_jit_aggregate_functions.sql) (in query: SELECT CounterID, min(WatchID), max(WatchID), sum(WatchID), avg(WatchID), avgWeighted(WatchID, CounterID), count(WatchID), groupBitOr(WatchID), groupBitAnd(WatchID), groupBitXor(WatchID) FROM test.hits GROUP BY CounterID ORDER BY count() DESC LIMIT 20;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:40.733946 [ 3999 ] {4fb413f5-eb75-45bc-bd53-43dd06616ba2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:43.890292 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:43.890489 [ 3999 ] {25660467-b5a3-4c96-955b-ecb076d1bfdf} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48276) (comment: 00079_array_join_not_used_joined_column.sql) (in query: SELECT PP.Key1 AS x1, ParsedParams.Key2 AS x2 FROM test.hits ARRAY JOIN ParsedParams AS PP WHERE CounterID = 1704509 ORDER BY x1, x2 LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:43.895706 [ 3999 ] {25660467-b5a3-4c96-955b-ecb076d1bfdf} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:45.128941 [ 3999 ] {8250e0cf-5ce8-4a7c-9919-010a2d338e58} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56958) (comment: 02555_davengers_rename_chain.sh) (in query: SELECT * FROM wrong_metadata_compact ORDER BY a1 FORMAT JSONEachRow), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:45.163105 [ 3999 ] {8250e0cf-5ce8-4a7c-9919-010a2d338e58} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:50.922998 [ 3949 ] {750ee08d-225a-4e2c-99bf-62f390b46173} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57038) (comment: 00151_order_by_read_in_order.sql) (in query: SELECT CounterID FROM test.hits ORDER BY CounterID DESC LIMIT 50;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:06:50.924443 [ 3949 ] {750ee08d-225a-4e2c-99bf-62f390b46173} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:06:53.354657 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:54.742811 [ 4733 ] {4a7fc94c-68ff-4311-ae50-52a22e8edb6a} <Error> AsynchronousInsertQueue: Failed parsing for query 'INSERT INTO test_13.t_async_inserts_flush FORMAT JSONEachRow' with query id 9463b319-91c5-4bdf-bb34-3760fb3f24b7. DB::Exception: Cannot parse input: expected '"' before: 'str"}': (while reading the value of key a): (at row 1)

2024.11.04 23:06:54.789270 [ 3999 ] {1c42e21a-f487-424a-bac7-121a418ab616} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59342) (comment: 02726_async_insert_flush_queue.sql) (in query: SELECT count() FROM t_async_inserts_flush;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:54.790963 [ 3999 ] {1c42e21a-f487-424a-bac7-121a418ab616} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:55.088622 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:56.126119 [ 3949 ] {72150a2f-5fe2-4d5b-b3b2-9aeaba3ae6d1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57072) (comment: 00095_hyperscan_profiler.sql) (in query: SELECT count() FROM test.hits WHERE multiFuzzyMatchAny(URL, 2, ['about/address', 'for_woman', '^https?://lm-company.ruy/$', 'ultimateguitar.com']);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:56.246657 [ 3949 ] {72150a2f-5fe2-4d5b-b3b2-9aeaba3ae6d1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:56.742421 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:06:57.855133 [ 3949 ] {d818fef1-ae07-4b05-83dd-c98b202ba991} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59376) (comment: 00024_random_counters.sql) (in query: SELECT uniq(UserID), sum(Sign) FROM test.visits WHERE CounterID = 32152608;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:06:57.858063 [ 3949 ] {d818fef1-ae07-4b05-83dd-c98b202ba991} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:00.944174 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:00.944441 [ 3949 ] {8a82c55c-e745-4655-8b48-f7336633a571} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59382) (comment: 00031_array_enumerate_uniq.sql) (in query: SELECT UserID, arrayEnumerateUniq(groupArray(SearchPhrase)) AS arr FROM ( SELECT UserID, SearchPhrase FROM test.hits WHERE CounterID = 1704509 AND UserID IN ( SELECT UserID FROM test.hits WHERE notEmpty(SearchPhrase) AND CounterID = 1704509 GROUP BY UserID HAVING count() > 1 ) ORDER BY UserID, WatchID ) WHERE notEmpty(SearchPhrase) GROUP BY UserID HAVING length(arr) > 1 ORDER BY UserID LIMIT 20 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:01.005038 [ 3949 ] {8a82c55c-e745-4655-8b48-f7336633a571} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:02.222444 [ 3949 ] {2153a794-dc6a-475f-ae0e-b51841625dcf} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59424) (comment: 00729_prewhere_array_join.sql) (in query: insert into t1_00729 (id,val,nid,eDate) values ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:07:02.223204 [ 3949 ] {2153a794-dc6a-475f-ae0e-b51841625dcf} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:07:05.574499 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:07.207506 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:10.162646 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f35a2ef5-e44d-4e42-94b5-c841517c6ce9::202411_1_16_3}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:10.502752 [ 3949 ] {f6d888ea-cace-4d23-b2fb-515290611241} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60010) (comment: 03217_filtering_in_system_tables.sql) (in query: -- StorageSystemReplicas
 SELECT argMax(read_rows, event_time_microseconds) FROM system.query_log WHERE 1 AND current_database = currentDatabase() AND query LIKE '%SELECT database, table, left(replica_name, 2) FROM system.replicas WHERE database = currentDatabase() AND table = \'test_03217_system_tables_replica_1\' AND replica_name LIKE \'r1\%\';' AND type = 'QueryFinish';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:10.506844 [ 3949 ] {f6d888ea-cace-4d23-b2fb-515290611241} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:11.190481 [ 4007 ] {c145c58d-d430-43f1-b3e8-c3df6ea64977} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52168) (comment: 00178_quantile_ddsketch.sql) (in query: SELECT `__table1`.`CounterID` AS `k`, round(quantileDD(0.01, 0.5)(`__table1`.`ResolutionWidth`), 2) AS `round(quantileDD(0.01, 0.5)(ResolutionWidth), 2)` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`CounterID` ORDER BY count() DESC, `__table1`.`CounterID` ASC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:11.520000 [ 4007 ] {c145c58d-d430-43f1-b3e8-c3df6ea64977} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:11.896006 [ 3999 ] {4d092640-ac51-4e6e-9cb5-3c7bb4e699a2} <Error> executeQuery: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
: While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59434) (comment: 00178_quantile_ddsketch.sql) (in query: SELECT CounterID AS k, round(quantileDD(0.01, 0.5)(ResolutionWidth), 2) FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY count() DESC, CounterID LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:07:11.899820 [ 3999 ] {4d092640-ac51-4e6e-9cb5-3c7bb4e699a2} <Error> TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
: While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:07:12.919815 [ 3885 ] {6ac7a2c3-6de5-4303-b5d2-dd04e7e253e2} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:34316). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34316) (comment: 00178_quantile_ddsketch.sql) (in query: SELECT `__table1`.`CounterID` AS `k`, round(quantileDD(0.01, 0.5)(`__table1`.`ResolutionWidth`), 2) AS `round(quantileDD(0.01, 0.5)(ResolutionWidth), 2)` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`CounterID` ORDER BY count() DESC, `__table1`.`CounterID` ASC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendEndOfStream() @ 0x000000003a4fe40b
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d73e9
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:13.577058 [ 3885 ] {6ac7a2c3-6de5-4303-b5d2-dd04e7e253e2} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:34316). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:14.975534 [ 3885 ] {6ac7a2c3-6de5-4303-b5d2-dd04e7e253e2} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:34316). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendEndOfStream() @ 0x000000003a4fe40b
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d73e9
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:14.975645 [ 3949 ] {586cc5b5-50b4-4abb-b421-d7eabaf728e7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60024) (comment: 00910_zookeeper_test_alter_compression_codecs_long.sql) (in query: INSERT INTO alter_compression_codec2 VALUES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp:0: DB::ReplicatedMergeTreeSinkImpl<false>::consume(DB::Chunk) @ 0x0000000039c30d94
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:07:14.975674 [ 3999 ] {fd880791-4fa0-4b40-9bdb-a55c135230ce} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60054) (comment: 00005_filtering.sql) (in query: SELECT count() FROM test.hits WHERE AdvEngineID != 0 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:14.976445 [ 3885 ] {6ac7a2c3-6de5-4303-b5d2-dd04e7e253e2} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:07:14.976831 [ 3949 ] {586cc5b5-50b4-4abb-b421-d7eabaf728e7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp:0: DB::ReplicatedMergeTreeSinkImpl<false>::consume(DB::Chunk) @ 0x0000000039c30d94
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:07:15.103893 [ 3999 ] {fd880791-4fa0-4b40-9bdb-a55c135230ce} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:15.678327 [ 3949 ] {00609775-f03c-4950-bddb-ae56a0fcc208} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33924) (comment: 00028_shard_big_agg_aj_distributed.sql) (in query: INSERT INTO big_array SELECT groupArray(number % 255) AS x FROM (SELECT * FROM system.numbers LIMIT 1000000);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:15.682573 [ 3949 ] {00609775-f03c-4950-bddb-ae56a0fcc208} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:17.333698 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:18.499606 [ 3999 ] {fdcca518-6c69-45c8-8144-e5ccf4a1fa2c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33944) (comment: 00011_sorting.sql) (in query: SELECT EventTime::DateTime('Asia/Dubai') FROM test.hits ORDER BY EventTime DESC LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:18.499649 [ 3949 ] {b69da5d3-83bc-48dc-af88-b345826343d9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33942) (comment: 02494_zero_copy_and_projection_and_mutation_work_together.sql) (in query: CREATE TABLE wikistat2 (`time` DateTime, `project` LowCardinality(String), `subproject` LowCardinality(String), `path` String, `hits` UInt64, PROJECTION total (SELECT project, subproject, path, sum(hits), count() GROUP BY project, subproject, path)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{database}/02494_zero_copy_and_projection', '2') ORDER BY (path, time) SETTINGS old_parts_lifetime = 1, cleanup_delay_period = 0, cleanup_delay_period_random_add = 0, cleanup_thread_preferred_points_per_iteration = 0, allow_remote_fs_zero_copy_replication = 1, min_bytes_for_wide_part = 0, index_granularity = 5538, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = false, max_file_name_length = 11, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 66172476, compact_parts_max_granules_to_buffer = 256, compact_parts_merge_max_bytes_to_prefetch_part = 13345685, merge_max_block_size = 15167, prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 8202974879, index_granularity_bytes = 15817319, concurrent_part_removal_threshold = 60, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = true, marks_compress_block_size = 41942, primary_key_compress_block_size = 87141), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::loadDataParts(bool, std::optional<std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>>) @ 0x000000003910bb4b
8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions, bool) @ 0x0000000038050e38
9. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageReplicatedMergeTree> std::allocate_shared[abi:v15000]<DB::StorageReplicatedMergeTree, std::allocator<DB::StorageReplicatedMergeTree>, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions&, bool&, void>(std::allocator<DB::StorageReplicatedMergeTree> const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, DB::RenamingRestrictions&, bool&) @ 0x0000000039d01af6
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2e7f
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
12. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:07:19.708081 [ 3999 ] {fdcca518-6c69-45c8-8144-e5ccf4a1fa2c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:19.708090 [ 3949 ] {b69da5d3-83bc-48dc-af88-b345826343d9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::loadDataParts(bool, std::optional<std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>>) @ 0x000000003910bb4b
8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions, bool) @ 0x0000000038050e38
9. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageReplicatedMergeTree> std::allocate_shared[abi:v15000]<DB::StorageReplicatedMergeTree, std::allocator<DB::StorageReplicatedMergeTree>, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions&, bool&, void>(std::allocator<DB::StorageReplicatedMergeTree> const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, DB::RenamingRestrictions&, bool&) @ 0x0000000039d01af6
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2e7f
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
12. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:07:19.763319 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_15_3}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:21.573913 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:21.574037 [ 3949 ] {515c9bd9-e6a8-4e0a-940c-561bcd2c0d29} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=831). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33946) (comment: 00158_cache_dictionary_has.sql) (in query: SELECT sum(flag) FROM (SELECT dictHas('db_dict.cache_hits', toUInt64(WatchID)) as flag FROM test.hits PREWHERE WatchID % 1400 == 0 LIMIT 100);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:07:21.586042 [ 3949 ] {515c9bd9-e6a8-4e0a-940c-561bcd2c0d29} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=831). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:07:24.578621 [ 3999 ] {4d5510c2-e47a-4008-915c-3b21c319970d} <Error> executeQuery: Code: 524. DB::Exception: Cannot ALTER column: Cannot apply mutation because it breaks projection p1. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33986) (comment: 02920_alter_column_of_projections.sql) (in query: -- Cannot ALTER, which breaks key column of projection.
 ALTER TABLE t MODIFY COLUMN age Nullable(Int32);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Storages/AlterCommands.cpp:1184: DB::AlterCommands::apply(DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context const>) const @ 0x000000003799c2bd
4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000003916bd91
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
8. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:07:24.579878 [ 3999 ] {4d5510c2-e47a-4008-915c-3b21c319970d} <Error> TCPHandler: Code: 524. DB::Exception: Cannot ALTER column: Cannot apply mutation because it breaks projection p1. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Storages/AlterCommands.cpp:1184: DB::AlterCommands::apply(DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context const>) const @ 0x000000003799c2bd
4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000003916bd91
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
8. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:07:24.640560 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:25.700450 [ 3999 ] {002c8fb0-d766-491a-a6d9-09ab881e7067} <Error> executeQuery: Code: 70. DB::Exception: Conversion from AggregateFunction(uniq, Int16) to AggregateFunction(uniq, Int32) is not supported: While processing _CAST(`uniq(i)`, 'AggregateFunction(uniq, Int32)') AS `uniq(i)`: Cannot apply mutation because it breaks projection p1. (CANNOT_CONVERT_TYPE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33986) (comment: 02920_alter_column_of_projections.sql) (in query: -- Cannot ALTER, uniq(Int16) is not compatible with uniq(Int32).
 ALTER TABLE t MODIFY COLUMN i Int32;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. DB::(anonymous namespace)::FunctionCast::prepareImpl(std::shared_ptr<DB::IDataType const> const&, std::shared_ptr<DB::IDataType const> const&, bool) const @ 0x0000000007f867af
4. DB::(anonymous namespace)::FunctionCast::prepareUnpackDictionaries(std::shared_ptr<DB::IDataType const> const&, std::shared_ptr<DB::IDataType const> const&) const @ 0x0000000007f6c3a9
5. DB::(anonymous namespace)::FunctionCast::prepare(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007f674ee
6. ./build_docker/./src/Interpreters/ActionsDAG.cpp:0: DB::ActionsDAG::addFunctionImpl(std::shared_ptr<DB::IFunctionBase const> const&, std::vector<DB::ActionsDAG::Node const*, std::allocator<DB::ActionsDAG::Node const*>>, std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>>, String, std::shared_ptr<DB::IDataType const>, bool) @ 0x000000003268e4ab
7. ./build_docker/./src/Interpreters/ActionsDAG.cpp:244: DB::ActionsDAG::addFunction(std::shared_ptr<DB::IFunctionOverloadResolver> const&, std::vector<DB::ActionsDAG::Node const*, std::allocator<DB::ActionsDAG::Node const*>>, String) @ 0x000000003268bb8b
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ScopeStack::addFunction(std::shared_ptr<DB::IFunctionOverloadResolver> const&, std::vector<String, std::allocator<String>> const&, String) @ 0x0000000032cebb6a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ActionsMatcher::Data::addFunction(std::shared_ptr<DB::IFunctionOverloadResolver> const&, std::vector<String, std::allocator<String>> const&, String) @ 0x0000000032d1f1b3
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf325a
11. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
12. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
13. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
14. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
15. ./build_docker/./src/Interpreters/inplaceBlockConversions.cpp:167: DB::(anonymous namespace)::createExpressions(DB::Block const&, std::shared_ptr<DB::IAST>, bool, std::shared_ptr<DB::Context const>) @ 0x00000000365f9c21
16. ./build_docker/./src/Interpreters/inplaceBlockConversions.cpp:181: DB::performRequiredConversions(DB::Block&, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const>) @ 0x00000000365f821f
17. ./contrib/llvm-project/libcxx/include/list:616: DB::AlterCommands::apply(DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context const>) const @ 0x000000003799a931
18. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000003916bd91
19. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
20. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
21. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:07:25.700489 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:25.701996 [ 3999 ] {002c8fb0-d766-491a-a6d9-09ab881e7067} <Error> TCPHandler: Code: 70. DB::Exception: Conversion from AggregateFunction(uniq, Int16) to AggregateFunction(uniq, Int32) is not supported: While processing _CAST(`uniq(i)`, 'AggregateFunction(uniq, Int32)') AS `uniq(i)`: Cannot apply mutation because it breaks projection p1. (CANNOT_CONVERT_TYPE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. DB::(anonymous namespace)::FunctionCast::prepareImpl(std::shared_ptr<DB::IDataType const> const&, std::shared_ptr<DB::IDataType const> const&, bool) const @ 0x0000000007f867af
4. DB::(anonymous namespace)::FunctionCast::prepareUnpackDictionaries(std::shared_ptr<DB::IDataType const> const&, std::shared_ptr<DB::IDataType const> const&) const @ 0x0000000007f6c3a9
5. DB::(anonymous namespace)::FunctionCast::prepare(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007f674ee
6. ./build_docker/./src/Interpreters/ActionsDAG.cpp:0: DB::ActionsDAG::addFunctionImpl(std::shared_ptr<DB::IFunctionBase const> const&, std::vector<DB::ActionsDAG::Node const*, std::allocator<DB::ActionsDAG::Node const*>>, std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>>, String, std::shared_ptr<DB::IDataType const>, bool) @ 0x000000003268e4ab
7. ./build_docker/./src/Interpreters/ActionsDAG.cpp:244: DB::ActionsDAG::addFunction(std::shared_ptr<DB::IFunctionOverloadResolver> const&, std::vector<DB::ActionsDAG::Node const*, std::allocator<DB::ActionsDAG::Node const*>>, String) @ 0x000000003268bb8b
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ScopeStack::addFunction(std::shared_ptr<DB::IFunctionOverloadResolver> const&, std::vector<String, std::allocator<String>> const&, String) @ 0x0000000032cebb6a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ActionsMatcher::Data::addFunction(std::shared_ptr<DB::IFunctionOverloadResolver> const&, std::vector<String, std::allocator<String>> const&, String) @ 0x0000000032d1f1b3
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf325a
11. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
12. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
13. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
14. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
15. ./build_docker/./src/Interpreters/inplaceBlockConversions.cpp:167: DB::(anonymous namespace)::createExpressions(DB::Block const&, std::shared_ptr<DB::IAST>, bool, std::shared_ptr<DB::Context const>) @ 0x00000000365f9c21
16. ./build_docker/./src/Interpreters/inplaceBlockConversions.cpp:181: DB::performRequiredConversions(DB::Block&, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const>) @ 0x00000000365f821f
17. ./contrib/llvm-project/libcxx/include/list:616: DB::AlterCommands::apply(DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context const>) const @ 0x000000003799a931
18. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000003916bd91
19. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
20. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
21. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:07:26.144833 [ 3203 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:07:26.144972 [ 3213 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:07:26.145051 [ 3210 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:07:26.145160 [ 3207 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:07:26.145326 [ 3215 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:07:27.188409 [ 3709 ] {149a5d8e-e96f-4282-a8dc-8003fa05f0aa::all_1_1_0_3} <Error> MutatePlainMergeTreeTask: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:27.218698 [ 3206 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:27.218743 [ 3211 ] {} <Error> test_13.wikistat1 (10be863e-8fe6-4772-b0e4-4fd3503bd58a): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/02494_zero_copy_and_projection/replicas/1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:27.967477 [ 3202 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:30.111151 [ 3209 ] {} <Error> test_13.wikistat1 (10be863e-8fe6-4772-b0e4-4fd3503bd58a): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:30.111344 [ 3218 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:31.052647 [ 3247 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3914: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x0000000038216241
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a61f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:31.052726 [ 3258 ] {} <Error> test_13.wikistat1 (10be863e-8fe6-4772-b0e4-4fd3503bd58a): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/02494_zero_copy_and_projection/replicas/1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3914: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x0000000038216241
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a61f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:32.754125 [ 3999 ] {1b69b4dd-546e-47b8-825a-a67795c2bf86} <Error> executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_3.txt' with part 'all_1_1_0_2' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33986) (comment: 02920_alter_column_of_projections.sql) (in query: -- Another ALTER to wait for.
 ALTER TABLE t MODIFY COLUMN j Int64 SETTINGS mutations_sync = 2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x0000000039d143c5
6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:07:34.341336 [ 3709 ] {149a5d8e-e96f-4282-a8dc-8003fa05f0aa::all_1_1_0_3} <Error> virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:587: DB::MutateAllPartColumnsTask::finalize() @ 0x000000003993a011
16. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000003992a32e
17. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:2010: DB::MutateTask::execute() @ 0x00000000398caa43
18. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:0: DB::MutatePlainMergeTreeTask::executeStep() @ 0x0000000039dd7d22
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:34.341344 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:34.341342 [ 3999 ] {1b69b4dd-546e-47b8-825a-a67795c2bf86} <Error> TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_3.txt' with part 'all_1_1_0_2' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x0000000039d143c5
6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:07:37.099249 [ 3999 ] {7fb30dcd-83e9-42e0-a565-2ec0e8efd0e9} <Error> executeQuery: Code: 206. DB::Exception: JOIN  (SELECT 1 AS A, 2 AS B FROM system.one) AS X ALL LEFT JOIN (SELECT 3 AS A, 2 AS B FROM system.one) USING (B) no alias for subquery or table function SELECT 3 AS A, 2 AS B. In scope SELECT * FROM (SELECT 1 AS A, 2 AS B) AS X ALL LEFT JOIN (SELECT 3 AS A, 2 AS B) USING (B) (set joined_subquery_requires_alias = 0 to disable restriction). (ALIAS_REQUIRED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32790) (comment: 01000_subquery_requires_alias.sql) (in query: SELECT * FROM (SELECT 1 as A, 2 as B) X ALL LEFT JOIN (SELECT 3 as A, 2 as B) USING (B);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x0000000007ecb9de
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2440: DB::(anonymous namespace)::QueryAnalyzer::validateJoinTableExpressionWithoutAlias(std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034881c19
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x0000000034874794
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5b6d
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:07:37.100395 [ 3999 ] {7fb30dcd-83e9-42e0-a565-2ec0e8efd0e9} <Error> TCPHandler: Code: 206. DB::Exception: JOIN  (SELECT 1 AS A, 2 AS B FROM system.one) AS X ALL LEFT JOIN (SELECT 3 AS A, 2 AS B FROM system.one) USING (B) no alias for subquery or table function SELECT 3 AS A, 2 AS B. In scope SELECT * FROM (SELECT 1 AS A, 2 AS B) AS X ALL LEFT JOIN (SELECT 3 AS A, 2 AS B) USING (B) (set joined_subquery_requires_alias = 0 to disable restriction). (ALIAS_REQUIRED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x0000000007ecb9de
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2440: DB::(anonymous namespace)::QueryAnalyzer::validateJoinTableExpressionWithoutAlias(std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034881c19
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::(anonymous namespace)::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x0000000034874794
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5b6d
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:07:37.917121 [ 3999 ] {5c125287-6aca-47af-a258-4167ddba0874} <Error> executeQuery: Code: 206. DB::Exception: JOIN  (SELECT 1 AS A, 2 AS B FROM system.one) ALL LEFT JOIN (SELECT 3 AS A, 2 AS B FROM system.one) AS Y USING (B) no alias for subquery or table function SELECT 1 AS A, 2 AS B. In scope SELECT * FROM (SELECT 1 AS A, 2 AS B) ALL LEFT JOIN (SELECT 3 AS A, 2 AS B) AS Y USING (B) (set joined_subquery_requires_alias = 0 to disable restriction). (ALIAS_REQUIRED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32790) (comment: 01000_subquery_requires_alias.sql) (in query: SELECT * FROM (SELECT 1 as A, 2 as B) ALL LEFT JOIN (SELECT 3 as A, 2 as B) Y USING (B);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x0000000007ecb9de
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2440: DB::(anonymous namespace)::QueryAnalyzer::validateJoinTableExpressionWithoutAlias(std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034881c19
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7731: DB::(anonymous namespace)::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x000000003487474b
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5b6d
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:07:37.917315 [ 3949 ] {6eb717cd-e92c-49ae-82b2-fbb7e16b9039} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=16): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32806) (comment: 00065_loyalty_with_storage_join.sql) (in query: INSERT INTO join SELECT UserID, toInt8(if((sum(SearchEngineID = 2) AS yandex) > (sum(SearchEngineID = 3) AS google), yandex / (yandex + google), -google / (yandex + google)) * 10) AS loyalty FROM hits WHERE (SearchEngineID = 2) OR (SearchEngineID = 3) GROUP BY UserID HAVING (yandex + google) > 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:37.918473 [ 3999 ] {5c125287-6aca-47af-a258-4167ddba0874} <Error> TCPHandler: Code: 206. DB::Exception: JOIN  (SELECT 1 AS A, 2 AS B FROM system.one) ALL LEFT JOIN (SELECT 3 AS A, 2 AS B FROM system.one) AS Y USING (B) no alias for subquery or table function SELECT 1 AS A, 2 AS B. In scope SELECT * FROM (SELECT 1 AS A, 2 AS B) ALL LEFT JOIN (SELECT 3 AS A, 2 AS B) AS Y USING (B) (set joined_subquery_requires_alias = 0 to disable restriction). (ALIAS_REQUIRED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x0000000007ecb9de
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:2440: DB::(anonymous namespace)::QueryAnalyzer::validateJoinTableExpressionWithoutAlias(std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034881c19
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7731: DB::(anonymous namespace)::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x000000003487474b
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5b6d
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:07:37.945788 [ 3949 ] {6eb717cd-e92c-49ae-82b2-fbb7e16b9039} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=37, jobs=16): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:40.149752 [ 3949 ] {daa43a73-ef09-441a-834d-a14388362ef3} <Error> executeQuery: Code: 10. DB::Exception: Column `v2` not found in table test_13.mv_extra_columns_dst (cb7c8f3b-1ee5-4d4d-9460-81c64fd0bee4). (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32814) (comment: 01019_materialized_view_select_extra_columns.sql) (in query: SELECT * FROM mv_extra_columns_view;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageSnapshot.cpp:214: DB::StorageSnapshot::getSampleBlockForColumns(std::vector<String, std::allocator<String>> const&) const @ 0x000000003865fde4
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::ReadFromMergeTree(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>>, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const> const&, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<Poco::Logger>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) @ 0x000000003b69f2ba
5. ./contrib/llvm-project/libcxx/include/vector:434: std::__unique_if<DB::ReadFromMergeTree>::__unique_single std::make_unique[abi:v15000]<DB::ReadFromMergeTree, std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&>(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&) @ 0x000000003946b1ef
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:921: DB::MergeTreeDataSelectExecutor::readFromParts(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) const @ 0x000000003942bebf
7. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:140: DB::MergeTreeDataSelectExecutor::read(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, bool) const @ 0x000000003942b247
8. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000039d0d26b
9. ./src/Processors/QueryPlan/QueryPlan.h:54: DB::StorageMaterializedView::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037edfb5e
10. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
11. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
12. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
13. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
14. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:07:40.149749 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:40.149864 [ 3999 ] {0199d70a-eaf1-463a-ad89-cc6e70a8e183} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=41): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32830) (comment: 00030_array_enumerate_uniq.sql) (in query: SELECT max(arrayJoin(arrayEnumerateUniq(arrayMap(x -> intDiv(x, 10), URLCategories)))) FROM test.hits ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:40.150613 [ 3949 ] {daa43a73-ef09-441a-834d-a14388362ef3} <Error> TCPHandler: Code: 10. DB::Exception: Column `v2` not found in table test_13.mv_extra_columns_dst (cb7c8f3b-1ee5-4d4d-9460-81c64fd0bee4). (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageSnapshot.cpp:214: DB::StorageSnapshot::getSampleBlockForColumns(std::vector<String, std::allocator<String>> const&) const @ 0x000000003865fde4
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::ReadFromMergeTree(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>>, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const> const&, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<Poco::Logger>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) @ 0x000000003b69f2ba
5. ./contrib/llvm-project/libcxx/include/vector:434: std::__unique_if<DB::ReadFromMergeTree>::__unique_single std::make_unique[abi:v15000]<DB::ReadFromMergeTree, std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&>(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&) @ 0x000000003946b1ef
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:921: DB::MergeTreeDataSelectExecutor::readFromParts(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) const @ 0x000000003942bebf
7. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:140: DB::MergeTreeDataSelectExecutor::read(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, bool) const @ 0x000000003942b247
8. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000039d0d26b
9. ./src/Processors/QueryPlan/QueryPlan.h:54: DB::StorageMaterializedView::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037edfb5e
10. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
11. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
12. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
13. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
14. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:07:40.199365 [ 3999 ] {0199d70a-eaf1-463a-ad89-cc6e70a8e183} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=41): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:42.721753 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:42.721869 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:42.721927 [ 3949 ] {dbe4539f-0a24-4b07-9e95-5a8180c6968f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32832) (comment: 00149_quantiles_timing_distributed.sql) (in query: SELECT sum(cityHash64(*)) FROM (SELECT CounterID, quantileTiming(0.5)(SendTiming), count() FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) WHERE SendTiming != -1 GROUP BY CounterID);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:42.752431 [ 3949 ] {dbe4539f-0a24-4b07-9e95-5a8180c6968f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:45.649501 [ 3707 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:47.008989 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.099385 [ 3949 ] {07096b98-cdda-4aac-8d92-b16631234104} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57902) (comment: 00146_aggregate_function_uniq.sql) (in query: SELECT RegionID, uniqHLL12(WatchID) AS X FROM remote('127.0.0.{1,2}', test, hits) GROUP BY RegionID HAVING X > 100000 ORDER BY RegionID ASC;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:48.099691 [ 3707 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.100440 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.114829 [ 3705 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.116166 [ 3705 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.116755 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.122667 [ 3700 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.124013 [ 3700 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.124645 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.130973 [ 3703 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.132334 [ 3703 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.132963 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.139666 [ 3702 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.192565 [ 3702 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.193192 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.199051 [ 3708 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.200499 [ 3708 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.201027 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.206366 [ 3709 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.207634 [ 3709 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.208182 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.213048 [ 3695 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.214324 [ 3695 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.214823 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.244574 [ 3949 ] {07096b98-cdda-4aac-8d92-b16631234104} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:07:48.455150 [ 3707 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.456646 [ 3707 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.457208 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.463127 [ 3705 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:48.464766 [ 3705 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.465419 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.587081 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {a7e77ef5-c915-489e-89ce-773df4eb75ed::202411_1_16_3}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:48.987783 [ 3949 ] {a07b05b7-fb0e-4b3e-ab6f-547aeb2154d1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57904) (comment: 00081_group_by_without_key_and_totals.sql) (in query: SELECT count() AS c FROM test.hits WHERE CounterID = 1704509 WITH TOTALS SETTINGS totals_mode = 'before_having', max_rows_to_group_by = 100000, group_by_overflow_mode = 'any';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:07:48.989585 [ 3949 ] {a07b05b7-fb0e-4b3e-ab6f-547aeb2154d1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:07:49.587159 [ 3693 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.589037 [ 3693 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.589801 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.595823 [ 3704 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.597315 [ 3704 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.597834 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.606257 [ 3696 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.607677 [ 3696 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.609524 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.618670 [ 3698 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.620892 [ 3698 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.621406 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.635648 [ 3694 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.640790 [ 3694 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.641324 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.707764 [ 3699 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.709441 [ 3699 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.710131 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.728809 [ 3703 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.730225 [ 3703 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.730820 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.820166 [ 3949 ] {6d0a0a4d-22f1-4110-975e-97517b68a3e9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57906) (comment: 00012_sorting_distributed.sql) (in query: SELECT EventTime::DateTime('Asia/Dubai') FROM remote('127.0.0.{1,2}', test, hits) ORDER BY EventTime DESC LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:49.820492 [ 3693 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.821837 [ 3693 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.822434 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.828811 [ 3708 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.830588 [ 3708 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.831152 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.845183 [ 3949 ] {6d0a0a4d-22f1-4110-975e-97517b68a3e9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:07:49.846772 [ 3709 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.847866 [ 3709 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.848439 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.853358 [ 3695 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.854526 [ 3695 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.855009 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.859798 [ 3694 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.861197 [ 3694 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.861645 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.866583 [ 3699 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.867873 [ 3699 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.868388 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.873594 [ 3703 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.874683 [ 3703 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.875097 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.981285 [ 3702 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.982393 [ 3702 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.982840 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.988778 [ 3700 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.990150 [ 3700 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.990698 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.996006 [ 3704 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:49.997286 [ 3704 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:49.997808 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.854146 [ 3701 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.855290 [ 3701 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.855722 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.862146 [ 3705 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.863215 [ 3705 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.863656 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.868638 [ 3697 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.869780 [ 3697 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.870226 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.875170 [ 3703 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.876241 [ 3703 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.876673 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.881499 [ 3693 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.882618 [ 3693 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.883067 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.887987 [ 3708 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.889204 [ 3708 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.889674 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.895879 [ 3707 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.897451 [ 3707 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.898098 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.913420 [ 3695 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:50.916210 [ 3695 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.917373 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:50.925633 [ 3694 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:52.724469 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:52.724699 [ 3694 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:52.725231 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:52.738494 [ 3702 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:52.739708 [ 3702 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:52.740290 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:52.746406 [ 3703 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:52.750709 [ 3703 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:52.751304 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.130658 [ 3693 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:53.131829 [ 3693 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.132555 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.138572 [ 3708 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:53.140001 [ 3708 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.140586 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.147481 [ 3707 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:53.149282 [ 3707 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.149840 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.162880 [ 3695 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:53.164722 [ 3695 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.165696 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.428108 [ 3703 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:53.429404 [ 3703 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.429958 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.438405 [ 3693 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:53.439709 [ 3693 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.440314 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:53.702850 [ 3708 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:55.052823 [ 3708 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.053649 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.054508 [ 3757 ] {} <Error> void DB::SystemLog<DB::SessionLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::SessionLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::flushImpl(std::vector<DB::SessionLogElement, std::allocator<DB::SessionLogElement>> const&, unsigned long) @ 0x000000003604be4a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::savingThreadFunction() @ 0x00000000360485a8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f4369c7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43689f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.114692 [ 3704 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:55.116006 [ 3704 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.140878 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.180787 [ 3707 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:55.182288 [ 3707 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.182993 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.848360 [ 3694 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:55.849565 [ 3694 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.850060 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.861363 [ 3703 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:55.862611 [ 3703 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.863126 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.868787 [ 3693 ] {} <Error> ReadWriteBufferFromHTTP: Failed to make request to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3' redirect to 'http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3'. Error: 'DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build))'. Failed at try 1/1.
2024.11.04 23:07:55.870079 [ 3693 ] {} <Error> d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1 (MergeFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:55.870588 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d293d425-d1fb-484d-adee-540a06c48467::all_0_0_1}: Code: 86. DB::HTTPException: Received error from remote server http://d7407d092bf4:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Fclickhouse%2Ftest_13%2Freplicas%2Fmaster&part=all_0_0_1&client_protocol_version=8&compress=false&remote_fs_metadata=s3. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.3.12.76.altinitystable (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, int, String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, int&&, String const&, String const&) @ 0x000000001f94a3bb
3. ./src/Common/Exception.h:44: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001f949ade
4. ./build_docker/./src/IO/HTTPCommon.cpp:84: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream<char, std::char_traits<char>>&, bool) @ 0x000000001f94955d
5. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&, bool) const @ 0x000000002545f00f
6. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional<DB::ReadWriteBufferFromHTTP::HTTPRange> const&) @ 0x000000002545f5af
7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:387: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000025461354
8. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:448: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ReadWriteBufferFromHTTP::nextImpl()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000025468f77
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::doWithRetries(std::function<void ()>&&, std::function<void ()>, bool) const @ 0x0000000025458dc0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000254655e3
11. DB::ReadBuffer::next() @ 0x0000000008ec3eef
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>, bool, bool, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>, bool, std::optional<DB::ReadWriteBufferFromHTTP::HTTPFileInfo>) @ 0x000000002545de03
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::ReadWriteBufferFromHTTP>::__unique_single std::make_unique[abi:v15000]<DB::ReadWriteBufferFromHTTP, DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function<void (std::basic_ostream<char, std::char_traits<char>>&)>&, bool&, bool&, std::vector<DB::HTTPHeaderEntry, std::allocator<DB::HTTPHeaderEntry>>&, bool&, std::nullopt_t const&) @ 0x000000002544b073
14. ./src/IO/ReadWriteBufferFromHTTP.h:0: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr<DB::Throttler>, bool, String const&, std::optional<DB::CurrentlySubmergingEmergingTagger>*, bool, std::shared_ptr<DB::IDisk>) @ 0x0000000038dae95b
15. ./contrib/llvm-project/libcxx/include/string:1499: std::shared_ptr<DB::IMergeTreeDataPart> std::__function::__policy_invoker<std::shared_ptr<DB::IMergeTreeDataPart> ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool)::$_4, std::shared_ptr<DB::IMergeTreeDataPart> ()>>(std::__function::__policy_storage const*) @ 0x0000000038426558
16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4918: DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x000000003818eefa
17. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x0000000038163c10
18. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:169: DB::ReplicatedMergeMutateTaskBase::executeImpl()::$_1::operator()(bool) const @ 0x0000000039a63c2f
19. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a6086c
20. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:56.196709 [ 4852 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/qwn/zujumduzsosmhuldizshcrckbqoee, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1a8d0701-881b-4e8c-aeae-00a82dba00db, upload has finished false.
2024.11.04 23:07:56.197162 [ 4852 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/qwn/zujumduzsosmhuldizshcrckbqoee, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1a8d0701-881b-4e8c-aeae-00a82dba00db, upload has finished false
2024.11.04 23:07:56.207274 [ 4852 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/qwn/zujumduzsosmhuldizshcrckbqoee, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1a8d0701-881b-4e8c-aeae-00a82dba00db, upload has finished false
2024.11.04 23:07:56.967169 [ 4221 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/tdj/ezymompkqndeuncmuglkknvfljuan, total size 78133164, count 78133164, hidden_size 0, offset 0, with pool: true, prefinalized true, finalized false, upload id f85137a9-3807-4143-93fb-82ba9d4e9f0f, upload has finished false.
2024.11.04 23:07:56.967451 [ 4221 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/tdj/ezymompkqndeuncmuglkknvfljuan, total size 78133164, count 78133164, hidden_size 0, offset 0, with pool: true, prefinalized true, finalized false, upload id f85137a9-3807-4143-93fb-82ba9d4e9f0f, upload has finished false
2024.11.04 23:07:56.977697 [ 4221 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/tdj/ezymompkqndeuncmuglkknvfljuan, total size 78133164, count 78133164, hidden_size 0, offset 0, with pool: true, prefinalized true, finalized false, upload id f85137a9-3807-4143-93fb-82ba9d4e9f0f, upload has finished false
2024.11.04 23:07:58.134915 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:07:59.194406 [ 3949 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57928) (comment: 00162_mmap_compression_none.sql) (in query: INSERT INTO hits_none SELECT Title FROM test.hits;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. ./src/IO/WriteBuffer.h:65: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039408090
17. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&) @ 0x00000000393f910f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f5c43
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::writeWithPermutation(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x0000000039885612
23. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039964471
24. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
25. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
26. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
27. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
28. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
29. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
30. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627

2024.11.04 23:07:59.248877 [ 3949 ] {f3f1b7c1-baa3-498d-a651-0c4770b9578c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. ./src/IO/WriteBuffer.h:65: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039408090
17. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&) @ 0x00000000393f910f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f5c43
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::writeWithPermutation(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x0000000039885612
23. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039964471
24. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
25. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
26. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
27. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
28. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
29. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
30. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627

2024.11.04 23:08:01.147690 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:02.063176 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_7_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/han/clsbupqxnbtkwciagquhmxsrybuap, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 75db3739-bf4c-4341-a818-af275b0c3498, upload has finished false.
2024.11.04 23:08:02.063432 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_7_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/han/clsbupqxnbtkwciagquhmxsrybuap, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 75db3739-bf4c-4341-a818-af275b0c3498, upload has finished false
2024.11.04 23:08:02.083540 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_7_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/han/clsbupqxnbtkwciagquhmxsrybuap, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 75db3739-bf4c-4341-a818-af275b0c3498, upload has finished false
2024.11.04 23:08:03.937555 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:03.937744 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_7_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 46 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:05.160698 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:05.961905 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_20_4}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:07.304207 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:07.304553 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:08.411740 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 13 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:09.807216 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=1): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 15 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:10.660968 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:12.498693 [ 3999 ] {51282e06-1ba4-4a6b-af6f-b951bbdb157a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57786) (comment: 01480_binary_operator_monotonicity.sql) (in query: INSERT INTO binary_op_mono5 VALUES ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:08:12.500327 [ 3999 ] {51282e06-1ba4-4a6b-af6f-b951bbdb157a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:08:14.412721 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:16.447484 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/mwl/hkdipwsltzrradpuvabqduqntbuqo, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 81d78843-ae79-4425-86a9-f8e8fb218ef9, upload has finished false.
2024.11.04 23:08:16.447601 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/mwl/hkdipwsltzrradpuvabqduqntbuqo, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 81d78843-ae79-4425-86a9-f8e8fb218ef9, upload has finished false
2024.11.04 23:08:16.455596 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/mwl/hkdipwsltzrradpuvabqduqntbuqo, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 81d78843-ae79-4425-86a9-f8e8fb218ef9, upload has finished false
2024.11.04 23:08:17.033950 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 47 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:17.036720 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:17.086381 [ 3999 ] {f3e1db4a-8135-4208-8369-1ae88dc7b2be} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=43, jobs=43). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50650) (comment: 01056_predicate_optimizer_bugs.sql) (in query: SELECT co,co2,co3,num FROM ( SELECT co,co2,co3,count() AS num FROM (SELECT dummy+1 AS co,dummy+2 AS co2 ,dummy+3 AS co3) GROUP BY cube (co,co2,co3) ) WHERE co!=0 AND co2 !=2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:17.089819 [ 3999 ] {f3e1db4a-8135-4208-8369-1ae88dc7b2be} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=43, jobs=43). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:17.315116 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:18.647303 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 9 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:21.082122 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 99 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:24.465419 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:25.797341 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:25.815487 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/clf/zoesxudqdppacnjyetfhvnzirwkqe, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a9844c17-8a73-4b47-bb1b-be791bc11e2a, upload has finished false.
2024.11.04 23:08:25.815586 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/clf/zoesxudqdppacnjyetfhvnzirwkqe, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a9844c17-8a73-4b47-bb1b-be791bc11e2a, upload has finished false
2024.11.04 23:08:25.845555 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/clf/zoesxudqdppacnjyetfhvnzirwkqe, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a9844c17-8a73-4b47-bb1b-be791bc11e2a, upload has finished false
2024.11.04 23:08:25.848692 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 52 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:26.273764 [ 3999 ] {d11d73ca-6583-47b6-8952-5647c50036ad} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50666) (comment: 00719_format_datetime_rand.sql) (in query: WITH toDate(today() + rand() % 4096) AS t SELECT count() FROM numbers(1000000) WHERE formatDateTime(t, '%F') != toString(t);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:26.276383 [ 3999 ] {d11d73ca-6583-47b6-8952-5647c50036ad} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:28.175909 [ 3999 ] {ca6cf76a-b8bb-4c32-a824-f68027ce0a2e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayDotProduct doesn't match: passed 1, should be 2: In scope SELECT arrayDotProduct([1, 2]). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47350) (comment: 02708_dotProduct.sql) (in query: SELECT arrayDotProduct([1, 2]);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, unsigned long&, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<unsigned long&>::type, std::type_identity<unsigned long&>::type>, String&&, unsigned long&, unsigned long&) @ 0x0000000027013230
3. ./build_docker/./src/Functions/IFunction.cpp:431: DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000002fd5694d
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd57d00
5. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
6. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
8. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
19. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:08:28.176873 [ 3999 ] {ca6cf76a-b8bb-4c32-a824-f68027ce0a2e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayDotProduct doesn't match: passed 1, should be 2: In scope SELECT arrayDotProduct([1, 2]). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, unsigned long&, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<unsigned long&>::type, std::type_identity<unsigned long&>::type>, String&&, unsigned long&, unsigned long&) @ 0x0000000027013230
3. ./build_docker/./src/Functions/IFunction.cpp:431: DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000002fd5694d
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd57d00
5. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
6. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
8. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
19. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:08:28.982111 [ 3999 ] {ce42c56c-fffe-4b43-b20a-782e78068d67} <Error> executeQuery: Code: 43. DB::Exception: Arguments for function arrayDotProduct must be of type Array: In scope SELECT arrayDotProduct([1, 2], 'abc'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47350) (comment: 02708_dotProduct.sql) (in query: SELECT arrayDotProduct([1, 2], 'abc');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce716c2
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:08:28.983494 [ 3999 ] {ce42c56c-fffe-4b43-b20a-782e78068d67} <Error> TCPHandler: Code: 43. DB::Exception: Arguments for function arrayDotProduct must be of type Array: In scope SELECT arrayDotProduct([1, 2], 'abc'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce716c2
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:08:28.997882 [ 3999 ] {88501adc-873a-457a-8be7-97a6e0f408ee} <Error> executeQuery: Code: 43. DB::Exception: Arguments for function arrayDotProduct must be of type Array: In scope SELECT arrayDotProduct('abc', [1, 2]). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47350) (comment: 02708_dotProduct.sql) (in query: SELECT arrayDotProduct('abc', [1, 2]);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce716c2
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:08:28.998637 [ 3999 ] {88501adc-873a-457a-8be7-97a6e0f408ee} <Error> TCPHandler: Code: 43. DB::Exception: Arguments for function arrayDotProduct must be of type Array: In scope SELECT arrayDotProduct('abc', [1, 2]). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce716c2
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:08:29.988248 [ 3999 ] {03d13de3-f2a7-47a9-b4bd-105978e6f122} <Error> executeQuery: Code: 43. DB::Exception: Function arrayDotProduct cannot process values of type String: In scope SELECT arrayDotProduct([1, 2], ['abc', 'def']). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47350) (comment: 02708_dotProduct.sql) (in query: SELECT arrayDotProduct([1, 2], ['abc', 'def']);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce7198a
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:08:29.989093 [ 3999 ] {03d13de3-f2a7-47a9-b4bd-105978e6f122} <Error> TCPHandler: Code: 43. DB::Exception: Function arrayDotProduct cannot process values of type String: In scope SELECT arrayDotProduct([1, 2], ['abc', 'def']). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce7198a
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:08:31.088414 [ 3999 ] {e8a0d604-d31b-4d7f-b4a4-3f80c0ed6e87} <Error> executeQuery: Code: 190. DB::Exception: Array arguments for function arrayDotProduct must have equal sizes: In scope SELECT arrayDotProduct([1, 2], [3, 4, 5]). (SIZES_OF_ARRAYS_DONT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47350) (comment: 02708_dotProduct.sql) (in query: SELECT arrayDotProduct([1, 2], [3, 4, 5]);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionArrayScalarProduct<DB::DotProduct>::executeWithResultTypeAndLeftTypeAndRightType<unsigned short, char8_t, char8_t>(COW<DB::IColumn>::immutable_ptr<DB::IColumn>, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, unsigned long) const @ 0x000000001cff3803
4. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionArrayScalarProduct<DB::DotProduct>::executeWithResultTypeAndLeftType<unsigned short, char8_t>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, unsigned long) const @ 0x000000001cfd87b0
5. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionArrayScalarProduct<DB::DotProduct>::executeWithResultType<unsigned short>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, unsigned long) const @ 0x000000001ce729aa
6. DB::FunctionArrayScalarProduct<DB::DotProduct>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x000000001ce6fe24
7. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
8. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
10. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
12. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
15. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
27. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:08:32.406998 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:32.407023 [ 3999 ] {e8a0d604-d31b-4d7f-b4a4-3f80c0ed6e87} <Error> TCPHandler: Code: 190. DB::Exception: Array arguments for function arrayDotProduct must have equal sizes: In scope SELECT arrayDotProduct([1, 2], [3, 4, 5]). (SIZES_OF_ARRAYS_DONT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionArrayScalarProduct<DB::DotProduct>::executeWithResultTypeAndLeftTypeAndRightType<unsigned short, char8_t, char8_t>(COW<DB::IColumn>::immutable_ptr<DB::IColumn>, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, unsigned long) const @ 0x000000001cff3803
4. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionArrayScalarProduct<DB::DotProduct>::executeWithResultTypeAndLeftType<unsigned short, char8_t>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, unsigned long) const @ 0x000000001cfd87b0
5. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionArrayScalarProduct<DB::DotProduct>::executeWithResultType<unsigned short>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, unsigned long) const @ 0x000000001ce729aa
6. DB::FunctionArrayScalarProduct<DB::DotProduct>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x000000001ce6fe24
7. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
8. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
10. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
12. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
15. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
19. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
27. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:08:33.168031 [ 3999 ] {5888307d-2d30-4709-8710-85f9b798f8b7} <Error> executeQuery: Code: 43. DB::Exception: Arguments for function arrayDotProduct must be of type Array: In scope SELECT dotProduct([1, 2], (3, 4, 5)). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47350) (comment: 02708_dotProduct.sql) (in query: SELECT dotProduct([1, 2], (3, 4, 5));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce716c2
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::TupleOrArrayFunction<DB::DotProduct>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001bb261ee
6. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
7. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
8. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
9. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:08:33.169246 [ 3999 ] {5888307d-2d30-4709-8710-85f9b798f8b7} <Error> TCPHandler: Code: 43. DB::Exception: Arguments for function arrayDotProduct must be of type Array: In scope SELECT dotProduct([1, 2], (3, 4, 5)). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::FunctionArrayScalarProduct<DB::DotProduct>::getReturnTypeImpl(std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&) const @ 0x000000001ce716c2
4. DB::IFunction::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000007c63f37
5. DB::TupleOrArrayFunction<DB::DotProduct>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001bb261ee
6. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
7. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
8. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
9. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:08:34.269827 [ 3344 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:34.269940 [ 3370 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:34.269979 [ 3362 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:34.270035 [ 3369 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:34.270119 [ 3366 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:34.270833 [ 3374 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:34.270936 [ 3368 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/1_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:34.271034 [ 3312 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:34.977978 [ 3371 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:34.979154 [ 3367 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:35.481629 [ 3735 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:08:35.482183 [ 3735 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:08:36.156727 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/las/pqmmcfoaahxskspdvgzyveakumxbn, total size 402653184, count 402653184, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 744a3d08-40be-477e-95f5-0bc2019a629f, upload has finished false.
2024.11.04 23:08:36.156827 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/las/pqmmcfoaahxskspdvgzyveakumxbn, total size 402653184, count 402653184, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 744a3d08-40be-477e-95f5-0bc2019a629f, upload has finished false
2024.11.04 23:08:36.259990 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/las/pqmmcfoaahxskspdvgzyveakumxbn, total size 402653184, count 402653184, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 744a3d08-40be-477e-95f5-0bc2019a629f, upload has finished false
2024.11.04 23:08:36.263261 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 72 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:38.986092 [ 3999 ] {d4c5b2f9-ab73-4af1-8fdf-3852b2235cf2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36624) (comment: 01896_jit_aggregation_function_if_long.sql) (in query: SELECT id, sumIf(value1, predicate_value), sumIf(value2, predicate_value), sumIf(value3, predicate_value), sumIf(value4, predicate_value) FROM test_table_signed_values GROUP BY id ORDER BY id;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:38.991867 [ 3999 ] {d4c5b2f9-ab73-4af1-8fdf-3852b2235cf2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:40.434162 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/wrn/circfyhiyxduyfbvssvxmrwydnvwf, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 24a540a1-fa93-4b9d-87ea-2722d9765d38, upload has finished false.
2024.11.04 23:08:40.434277 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/wrn/circfyhiyxduyfbvssvxmrwydnvwf, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 24a540a1-fa93-4b9d-87ea-2722d9765d38, upload has finished false
2024.11.04 23:08:40.445294 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/wrn/circfyhiyxduyfbvssvxmrwydnvwf, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 24a540a1-fa93-4b9d-87ea-2722d9765d38, upload has finished false
2024.11.04 23:08:41.048793 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 5 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:41.048830 [ 3999 ] {82ddf5e9-ac12-4271-9f06-226869752517} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36634) (comment: 02943_tokenbf_and_ngrambf_indexes_support_match_function.sql) (in query: CREATE TABLE ngrambf_tab (`id` UInt32, `str` String, INDEX idx str TYPE ngrambf_v1(3, 256, 2, 0) GRANULARITY 1) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 1, min_bytes_for_wide_part = 0, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = true, max_file_name_length = 128, min_bytes_for_full_part_storage = 536870912, compact_parts_max_bytes_to_buffer = 14718149, compact_parts_max_granules_to_buffer = 33, compact_parts_merge_max_bytes_to_prefetch_part = 28290221, merge_max_block_size = 5957, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 10737418240, index_granularity_bytes = 29637543, concurrent_part_removal_threshold = 100, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 47076, primary_key_compress_block_size = 20238), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::finalizeImpl() @ 0x0000000032472254
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:863: ? @ 0x00000000324714f9
14. DB::WriteBuffer::finalize() @ 0x0000000007c88f2e
15. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:307: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000390bcec2
16. ./build_docker/./src/Storages/StorageMergeTree.cpp:120: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04626
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
19. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
20. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
21. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
22. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
23. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:08:41.050651 [ 3999 ] {82ddf5e9-ac12-4271-9f06-226869752517} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::finalizeImpl() @ 0x0000000032472254
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:863: ? @ 0x00000000324714f9
14. DB::WriteBuffer::finalize() @ 0x0000000007c88f2e
15. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:307: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000390bcec2
16. ./build_docker/./src/Storages/StorageMergeTree.cpp:120: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04626
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
19. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
20. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
21. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
22. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
23. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:08:43.091693 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:43.917771 [ 3948 ] {555a1a59-ddaf-4d7a-857a-a139310061d7} <Error> executeQuery: Code: 396. DB::Exception: Limit for result exceeded, max rows: 400.00 thousand, current rows: 420.00 thousand. (TOO_MANY_ROWS_OR_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48020) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 111222333444 FORMAT TSV), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:08:43.918000 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_48_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:43.924444 [ 3948 ] {555a1a59-ddaf-4d7a-857a-a139310061d7} <Error> DynamicQueryHandler: Code: 396. DB::Exception: Limit for result exceeded, max rows: 400.00 thousand, current rows: 420.00 thousand. (TOO_MANY_ROWS_OR_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:44.549705 [ 3948 ] {cba3a518-ebc0-440e-9cca-4fecb3be6d16} <Error> executeQuery: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1000.00 B, current bytes: 1001.00 B. (TOO_MANY_ROWS_OR_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53222) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1001 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:08:44.553273 [ 3948 ] {cba3a518-ebc0-440e-9cca-4fecb3be6d16} <Error> DynamicQueryHandler: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1000.00 B, current bytes: 1001.00 B. (TOO_MANY_ROWS_OR_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:44.599424 [ 3948 ] {f175f808-4520-47ef-96ca-7e4a65775cd1} <Error> executeQuery: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1000.00 B, current bytes: 1001.00 B. (TOO_MANY_ROWS_OR_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53238) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1001 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:08:44.602207 [ 3948 ] {f175f808-4520-47ef-96ca-7e4a65775cd1} <Error> DynamicQueryHandler: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1000.00 B, current bytes: 1001.00 B. (TOO_MANY_ROWS_OR_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:45.627180 [ 3948 ] {b4acedb2-dae8-4874-ae7b-1fcd913d6bd7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=3): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53254) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1048577 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:08:45.657843 [ 3948 ] {b4acedb2-dae8-4874-ae7b-1fcd913d6bd7} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=3): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:45.784887 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/zll/vatzzbuaofismtxmropethikbwtpa, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a676a1fe-bf58-4226-a01d-95927a9ce1aa, upload has finished false.
2024.11.04 23:08:45.785020 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/zll/vatzzbuaofismtxmropethikbwtpa, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a676a1fe-bf58-4226-a01d-95927a9ce1aa, upload has finished false
2024.11.04 23:08:45.794734 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/zll/vatzzbuaofismtxmropethikbwtpa, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a676a1fe-bf58-4226-a01d-95927a9ce1aa, upload has finished false
2024.11.04 23:08:45.798307 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 39 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:46.239191 [ 3948 ] {2de21f76-4f97-4a3d-9649-1faf64a79752} <Error> executeQuery: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1.00 MiB, current bytes: 1.00 MiB. (TOO_MANY_ROWS_OR_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53256) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1048577 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:08:46.244997 [ 3948 ] {2de21f76-4f97-4a3d-9649-1faf64a79752} <Error> DynamicQueryHandler: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1.00 MiB, current bytes: 1.00 MiB. (TOO_MANY_ROWS_OR_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:46.638710 [ 3948 ] {e6cd2fce-5b3a-4a69-86c4-f1fe977bd476} <Error> executeQuery: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1.43 MiB, current bytes: 1.43 MiB. (TOO_MANY_ROWS_OR_BYTES) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53266) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1500001 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:08:46.647587 [ 3948 ] {e6cd2fce-5b3a-4a69-86c4-f1fe977bd476} <Error> DynamicQueryHandler: Code: 396. DB::Exception: Limit for result exceeded, max bytes: 1.43 MiB, current bytes: 1.43 MiB. (TOO_MANY_ROWS_OR_BYTES), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, ReadableSize, ReadableSize>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<ReadableSize>::type, std::type_identity<ReadableSize>::type>, char const*&, ReadableSize&&, ReadableSize&&) @ 0x0000000031926277
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:36: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x000000003192567c
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/Transforms/LimitsCheckingTransform.cpp:45: DB::LimitsCheckingTransform::transform(DB::Chunk&) @ 0x000000003b03e52b
6. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
7. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
8. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
9. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
12. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
15. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
16. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:46.763580 [ 3948 ] {c51b9b0b-7494-40a2-95c5-894bff0b7b48} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53270) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1500001 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:08:46.769345 [ 3948 ] {c51b9b0b-7494-40a2-95c5-894bff0b7b48} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:46.895293 [ 3948 ] {d3ad776b-12a4-4c27-af6b-354fa3605cb2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53280) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 5000000 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:08:46.900084 [ 3948 ] {d3ad776b-12a4-4c27-af6b-354fa3605cb2} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:47.403344 [ 3948 ] {574cac48-aa14-4084-9c67-29b4f695a6bc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=4): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53284) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 5000000 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:08:47.410535 [ 3948 ] {574cac48-aa14-4084-9c67-29b4f695a6bc} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=4): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:47.679023 [ 3948 ] {b236690f-fae9-413a-ad49-8be29aa3e36c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=4): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53286) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 5000000 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:08:47.684710 [ 3948 ] {b236690f-fae9-413a-ad49-8be29aa3e36c} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=4): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.h:0: DB::ParallelFormattingOutputFormat::consume(DB::Chunk) @ 0x000000003ac032a9
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c2048
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:48.688179 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 84 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:52.012529 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:52.016412 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {54259a93-f7ca-4ac1-8b7b-0c01fa06532c::202411_1_31_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:52.655867 [ 3460 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:52.655946 [ 3470 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:52.656008 [ 3448 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:52.656193 [ 3458 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:52.656200 [ 3459 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:08:52.656789 [ 3466 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:53.976236 [ 3467 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:53.976638 [ 3463 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:53.976693 [ 3461 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:53.976716 [ 3462 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:54.219864 [ 3949 ] {147d6ec6-11b1-42c4-8b69-50025180fe3c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43570) (comment: 00159_parallel_formatting_csv_and_friends.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c Format CSVWithNames), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:54.225995 [ 3949 ] {147d6ec6-11b1-42c4-8b69-50025180fe3c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:55.143179 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_31_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:56.177806 [ 3949 ] {fb2584d5-e9a6-4bf1-b4fc-d417f0ff9ffe} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=882). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43584) (comment: 00429_long_http_bufferization.sh) (in query: SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 500000 FORMAT RowBinary), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:08:56.179635 [ 3949 ] {fb2584d5-e9a6-4bf1-b4fc-d417f0ff9ffe} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=882). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:08:57.410098 [ 4776 ] {d69495ac-ca2a-4dab-be64-c24f97899ebc} <Error> void DB::ParallelParsingInputFormat::onBackgroundException(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac092c4
7. ./build_docker/./src/Processors/Formats/Impl/ParallelParsingInputFormat.cpp:53: DB::ParallelParsingInputFormat::segmentatorThreadFunction(std::shared_ptr<DB::ThreadGroup>) @ 0x000000003ac04bfc
8. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000003ac11416
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003ac1111f
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:57.413450 [ 3949 ] {d69495ac-ca2a-4dab-be64-c24f97899ebc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0): While executing ParallelParsingBlockInputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43616) (comment: 02784_schema_inference_null_as_default.sql) (in query: select * from format(JSONEachRow, '{"x" : null}, {"x" : 42}') settings schema_inference_make_columns_nullable=0, input_format_null_as_default=1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac092c4
7. ./build_docker/./src/Processors/Formats/Impl/ParallelParsingInputFormat.cpp:53: DB::ParallelParsingInputFormat::segmentatorThreadFunction(std::shared_ptr<DB::ThreadGroup>) @ 0x000000003ac04bfc
8. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000003ac11416
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003ac1111f
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:57.415179 [ 3949 ] {d69495ac-ca2a-4dab-be64-c24f97899ebc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0): While executing ParallelParsingBlockInputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac092c4
7. ./build_docker/./src/Processors/Formats/Impl/ParallelParsingInputFormat.cpp:53: DB::ParallelParsingInputFormat::segmentatorThreadFunction(std::shared_ptr<DB::ThreadGroup>) @ 0x000000003ac04bfc
8. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000003ac11416
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003ac1111f
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:08:57.743832 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:08:58.168137 [ 3739 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:08:58.168390 [ 3739 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:08:58.635725 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ypg/xfvunsefkknkhirgcgasgbnqhyxlf, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c8295c0c-65ee-4e6e-b205-f7b594d8dfb6, upload has finished false.
2024.11.04 23:08:58.635881 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ypg/xfvunsefkknkhirgcgasgbnqhyxlf, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c8295c0c-65ee-4e6e-b205-f7b594d8dfb6, upload has finished false
2024.11.04 23:08:58.694634 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ypg/xfvunsefkknkhirgcgasgbnqhyxlf, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c8295c0c-65ee-4e6e-b205-f7b594d8dfb6, upload has finished false
2024.11.04 23:08:58.787864 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 24 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:00.242196 [ 3949 ] {56a18e53-eb0f-4479-8f80-9e3f9aac3a95} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43650) (comment: 02377_modify_column_from_nested.sql) (in query: INSERT INTO t_nested_modify VALUES ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:09:01.706197 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:01.706234 [ 3949 ] {56a18e53-eb0f-4479-8f80-9e3f9aac3a95} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:09:01.706400 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:08.800596 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:09:08.800690 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:09:08.838338 [ 7091 ] {cbbefdec-c738-41d6-a69a-959bc19e5beb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=882). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43260) (comment: 01161_all_system_tables.sh) (in query: SELECT * FROM system.databases LIMIT 10000 SETTINGS allow_introspection_functions = 1 FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:08.839431 [ 7091 ] {cbbefdec-c738-41d6-a69a-959bc19e5beb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=882). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:09.198388 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/pcu/bpaiekqxuwmewunbonbvuacuxcpii, total size 301989888, count 301989888, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 10ab9f67-fda3-4f99-994d-ff4292344012, upload has finished false.
2024.11.04 23:09:09.198571 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/pcu/bpaiekqxuwmewunbonbvuacuxcpii, total size 301989888, count 301989888, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 10ab9f67-fda3-4f99-994d-ff4292344012, upload has finished false
2024.11.04 23:09:09.243685 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/pcu/bpaiekqxuwmewunbonbvuacuxcpii, total size 301989888, count 301989888, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 10ab9f67-fda3-4f99-994d-ff4292344012, upload has finished false
2024.11.04 23:09:09.247178 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 38 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:09.947342 [ 4775 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:09.948198 [ 3625 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:09.948304 [ 3617 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:09.948488 [ 3307 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:09.948861 [ 3112 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:09.949018 [ 3578 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:10.369667 [ 3619 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:10.369707 [ 3626 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:10.369820 [ 3529 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:11.104043 [ 3619 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
13. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
15. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
16. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:12.921756 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:12.921754 [ 7099 ] {8b04dd90-1972-4889-927f-7d0d41504216} <Error> executeQuery: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43470) (comment: 01161_all_system_tables.sh) (in query: SELECT * FROM system.part_moves_between_shards LIMIT 10000 SETTINGS allow_introspection_functions = 1 FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:240: zkutil::ZooKeeper::getChildren(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039a2c
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PartMovesBetweenShardsOrchestrator::syncStateFromZK() @ 0x0000000039a0253c
11. ./build_docker/./src/Storages/MergeTree/PartMovesBetweenShardsOrchestrator.cpp:0: DB::PartMovesBetweenShardsOrchestrator::getEntries() @ 0x0000000039a3bd65
12. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:7038: DB::StorageReplicatedMergeTree::getPartMovesBetweenShardsEntries() @ 0x00000000382f2ef5
13. ./contrib/llvm-project/libcxx/include/vector:1420: DB::StorageSystemPartMovesBetweenShards::fillData(std::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn>>>&, std::shared_ptr<DB::Context const>, DB::ActionsDAG::Node const*, std::vector<char8_t, std::allocator<char8_t>>) const @ 0x000000002fc319b1
14. ./contrib/llvm-project/libcxx/include/vector:434: DB::ReadFromSystemOneBlock::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002f7608d6
15. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:09:12.921837 [ 7090 ] {04938e89-28ef-427a-b578-4593450bc0b3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=885). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43612) (comment: 01161_all_system_tables.sh) (in query: SELECT * FROM information_schema.schemata LIMIT 10000 SETTINGS allow_introspection_functions = 1 FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:12.922885 [ 7099 ] {8b04dd90-1972-4889-927f-7d0d41504216} <Error> TCPHandler: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:240: zkutil::ZooKeeper::getChildren(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039a2c
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PartMovesBetweenShardsOrchestrator::syncStateFromZK() @ 0x0000000039a0253c
11. ./build_docker/./src/Storages/MergeTree/PartMovesBetweenShardsOrchestrator.cpp:0: DB::PartMovesBetweenShardsOrchestrator::getEntries() @ 0x0000000039a3bd65
12. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:7038: DB::StorageReplicatedMergeTree::getPartMovesBetweenShardsEntries() @ 0x00000000382f2ef5
13. ./contrib/llvm-project/libcxx/include/vector:1420: DB::StorageSystemPartMovesBetweenShards::fillData(std::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn>>>&, std::shared_ptr<DB::Context const>, DB::ActionsDAG::Node const*, std::vector<char8_t, std::allocator<char8_t>>) const @ 0x000000002fc319b1
14. ./contrib/llvm-project/libcxx/include/vector:434: DB::ReadFromSystemOneBlock::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002f7608d6
15. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:09:12.924380 [ 7090 ] {04938e89-28ef-427a-b578-4593450bc0b3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=885). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:16.130267 [ 3885 ] {23d1b29d-4e6e-4487-9763-096e7bc2eb9e} <Error> StatusRequestsPool: Error scheduling get status task for table test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/System/StorageSystemReplicas.cpp:0: DB::StatusRequestsPool::scheduleRequests(unsigned long, std::shared_ptr<DB::QueryStatus>) @ 0x000000002fb80893
7. ./build_docker/./src/Storages/System/StorageSystemReplicas.cpp:427: DB::ReadFromSystemReplicas::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002fb710d8
8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:16.221427 [ 3885 ] {23d1b29d-4e6e-4487-9763-096e7bc2eb9e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50878) (comment: 01161_all_system_tables.sh) (in query: SELECT * FROM system.replicas LIMIT 10000 SETTINGS allow_introspection_functions = 1 FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/System/StorageSystemReplicas.cpp:0: DB::StatusRequestsPool::scheduleRequests(unsigned long, std::shared_ptr<DB::QueryStatus>) @ 0x000000002fb80893
7. ./build_docker/./src/Storages/System/StorageSystemReplicas.cpp:427: DB::ReadFromSystemReplicas::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002fb710d8
8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:09:16.222558 [ 3885 ] {23d1b29d-4e6e-4487-9763-096e7bc2eb9e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/System/StorageSystemReplicas.cpp:0: DB::StatusRequestsPool::scheduleRequests(unsigned long, std::shared_ptr<DB::QueryStatus>) @ 0x000000002fb80893
7. ./build_docker/./src/Storages/System/StorageSystemReplicas.cpp:427: DB::ReadFromSystemReplicas::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002fb710d8
8. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
9. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:09:18.727380 [ 3885 ] {b8b69b88-00cc-419b-a488-58da6222211b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=887). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51070) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: SELECT URLRegions as d, toTimeZone(ClientEventTime, 'Asia/Dubai') as a, MobilePhoneModel as b, ParamPrice as e, ClientIP6 as c FROM test.hits LIMIT 5000 Format JSONCompactEachRowWithNamesAndTypes), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:18.729361 [ 3885 ] {b8b69b88-00cc-419b-a488-58da6222211b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=887). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:18.985064 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/pqj/yeeaggvbcuyymzebqukcmmyyupdag, total size 301989888, count 301989888, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d6fa24b7-63b8-418e-b231-7a246d3d532d, upload has finished false.
2024.11.04 23:09:18.985197 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/pqj/yeeaggvbcuyymzebqukcmmyyupdag, total size 301989888, count 301989888, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d6fa24b7-63b8-418e-b231-7a246d3d532d, upload has finished false
2024.11.04 23:09:19.028444 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/pqj/yeeaggvbcuyymzebqukcmmyyupdag, total size 301989888, count 301989888, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d6fa24b7-63b8-418e-b231-7a246d3d532d, upload has finished false
2024.11.04 23:09:19.033144 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 37 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:23.766914 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/loi/hihfcwwjajowvmkfjvkyfjughdcea, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 988ba4af-0e14-42fe-b8c4-63b9e39b3397, upload has finished false.
2024.11.04 23:09:23.767030 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/loi/hihfcwwjajowvmkfjvkyfjughdcea, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 988ba4af-0e14-42fe-b8c4-63b9e39b3397, upload has finished false
2024.11.04 23:09:23.773167 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/loi/hihfcwwjajowvmkfjvkyfjughdcea, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 988ba4af-0e14-42fe-b8c4-63b9e39b3397, upload has finished false
2024.11.04 23:09:23.778005 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 26 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:25.325727 [ 3949 ] {d054c0fe-d612-46e0-a2a6-39aad82604b0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55034) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: SELECT URLRegions as d, toTimeZone(ClientEventTime, 'Asia/Dubai') as a, MobilePhoneModel as b, ParamPrice as e, ClientIP6 as c FROM test.hits LIMIT 5000 Format JSONCompactStringsEachRowWithNamesAndTypes), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:09:25.328622 [ 3949 ] {d054c0fe-d612-46e0-a2a6-39aad82604b0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:09:25.389113 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 28 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:25.854323 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:27.871931 [ 3885 ] {e961d124-9ab2-438c-a959-6361a58a2c67} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55082) (comment: 00174_distinct_in_order.sql) (in query: insert into distinct_in_order select distinct CounterID, EventDate from test.hits order by CounterID, EventDate settings optimize_distinct_in_order=1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:27.879180 [ 3885 ] {e961d124-9ab2-438c-a959-6361a58a2c67} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:29.255916 [ 3885 ] {0faf06e3-651b-4957-8544-0ab2e4ecf8e6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55108) (comment: 00071_merge_tree_optimize_aio.sql) (in query: INSERT INTO test.hits_snippet(EventTime, EventDate, CounterID, UserID, URL, Referer) SELECT EventTime, EventDate, CounterID, UserID, URL, Referer FROM test.hits WHERE EventDate = toDate('2014-03-18') ORDER BY EventTime, EventDate, CounterID, UserID, URL, Referer ASC LIMIT 50;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:09:29.274934 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:30.558918 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:30.558950 [ 3885 ] {0faf06e3-651b-4957-8544-0ab2e4ecf8e6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:09:30.741318 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/bit/xjkrkrymgxynrwxjzqfhtxdmimkvb, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3253abee-0d1a-4dba-9e99-42bd27508e57, upload has finished false.
2024.11.04 23:09:30.741436 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/bit/xjkrkrymgxynrwxjzqfhtxdmimkvb, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3253abee-0d1a-4dba-9e99-42bd27508e57, upload has finished false
2024.11.04 23:09:30.763105 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/bit/xjkrkrymgxynrwxjzqfhtxdmimkvb, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3253abee-0d1a-4dba-9e99-42bd27508e57, upload has finished false
2024.11.04 23:09:30.768980 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 47 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:31.980611 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:33.324150 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:34.990276 [ 3739 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:09:34.990358 [ 3739 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:09:35.286040 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/zul/vfvvfgixngvvdvuanptsktcexvhph, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 7d9adec6-11f2-4033-ba94-4facd97d4b9f, upload has finished false.
2024.11.04 23:09:35.286201 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/zul/vfvvfgixngvvdvuanptsktcexvhph, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 7d9adec6-11f2-4033-ba94-4facd97d4b9f, upload has finished false
2024.11.04 23:09:35.329356 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/zul/vfvvfgixngvvdvuanptsktcexvhph, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 7d9adec6-11f2-4033-ba94-4facd97d4b9f, upload has finished false
2024.11.04 23:09:35.362451 [ 7091 ] {d6e33688-7e0f-4621-89ef-6ed72d4e8b8c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36672) (comment: 01161_all_system_tables.sh) (in query: SELECT * FROM system.zookeeper_log LIMIT 10000 SETTINGS allow_introspection_functions = 1 FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:09:35.362550 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 33 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:35.365894 [ 7091 ] {d6e33688-7e0f-4621-89ef-6ed72d4e8b8c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:09:37.915751 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:37.915861 [ 7091 ] {067c4681-8704-4a18-b79b-85272874ee5b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36694) (comment: 00160_decode_xml_component.sql) (in query: SELECT sum(DISTINCT sipHash64(decodeXMLComponent(Title) AS decoded)) FROM test.hits WHERE Title != decoded;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:38.813396 [ 7091 ] {067c4681-8704-4a18-b79b-85272874ee5b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:38.843183 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationNullable::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c6ace1
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:39.452358 [ 3885 ] {bcc9c7da-a49a-4fe5-9d41-96a9c8d32dd6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36716) (comment: 00075_left_array_join.sql) (in query: SELECT UserID, EventTime::DateTime('Asia/Dubai'), pp.Key1, pp.Key2, ParsedParams.Key1 FROM test.hits ARRAY JOIN ParsedParams AS pp WHERE CounterID = 1704509 ORDER BY UserID, EventTime, pp.Key1, pp.Key2 LIMIT 100;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:09:39.453969 [ 3885 ] {bcc9c7da-a49a-4fe5-9d41-96a9c8d32dd6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:09:39.935401 [ 7090 ] {ae2d280b-1f8f-4b0a-8005-4d5222d9010a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=889). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36766) (comment: 02124_buffer_with_type_map_long.sh) (in query: INSERT INTO t_buffer_map SELECT (range(10), range(10)), (range(10), range(10)) from numbers(100)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:39.935449 [ 7099 ] {5d35c6e6-8e19-49df-b2e4-5f48465b0dec} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36718) (comment: 02124_buffer_with_type_map_long.sh) (in query: INSERT INTO t_buffer_map SELECT (range(10), range(10)), (range(10), range(10)) from numbers(100)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:39.937479 [ 7090 ] {ae2d280b-1f8f-4b0a-8005-4d5222d9010a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=889). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:39.937514 [ 7099 ] {5d35c6e6-8e19-49df-b2e4-5f48465b0dec} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:40.006537 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ynb/firvaqjvecxqvncobhngdysyqdgjd, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f7e50f87-303e-47cb-96c2-9de6e99ff4f8, upload has finished false.
2024.11.04 23:09:40.006652 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ynb/firvaqjvecxqvncobhngdysyqdgjd, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f7e50f87-303e-47cb-96c2-9de6e99ff4f8, upload has finished false
2024.11.04 23:09:40.053827 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ynb/firvaqjvecxqvncobhngdysyqdgjd, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f7e50f87-303e-47cb-96c2-9de6e99ff4f8, upload has finished false
2024.11.04 23:09:40.057790 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 41 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:40.672576 [ 6987 ] {85555a6f-1869-41f4-9721-04cab9994b0b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=889). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36768) (comment: 02124_buffer_with_type_map_long.sh) (in query: SELECT * FROM t_buffer_map), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:40.673772 [ 6987 ] {85555a6f-1869-41f4-9721-04cab9994b0b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=889). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:41.589042 [ 7099 ] {a4439a9e-da0a-459f-b9da-df370796d9e0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36828) (comment: 00153_aggregate_arena_race.sql) (in query: insert into dest00153 select groupUniqArrayState(RefererDomain) from test.hits group by URLDomain;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:41.623959 [ 7099 ] {a4439a9e-da0a-459f-b9da-df370796d9e0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:42.735532 [ 7099 ] {9b81a277-9655-4142-ad62-d710b7760a61} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36846) (comment: 00042_any_left_join.sql) (in query: SELECT EventDate, hits, visits FROM ( SELECT EventDate, count() AS hits FROM test.hits GROUP BY EventDate ) ANY LEFT JOIN ( SELECT StartDate AS EventDate, sum(Sign) AS visits FROM test.visits GROUP BY EventDate ) USING EventDate ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:09:42.739229 [ 7099 ] {9b81a277-9655-4142-ad62-d710b7760a61} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:09:42.758306 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column zxid): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_1_21_4/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:637: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForOneColumn() const @ 0x000000003905f636
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:869: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b261
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
29. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:42.786030 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 88 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:43.145757 [ 3949 ] {982d08e1-eecb-434a-8eb4-7970be8a625e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36736) (comment: 02124_buffer_with_type_map_long.sh) (in query: SELECT * FROM t_buffer_map), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:43.146983 [ 3949 ] {982d08e1-eecb-434a-8eb4-7970be8a625e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:43.294047 [ 7238 ] {323093a2-0add-4d47-9bc8-60a1928812dc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36746) (comment: 02124_buffer_with_type_map_long.sh) (in query: SELECT * FROM t_buffer_map), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:43.294992 [ 7238 ] {323093a2-0add-4d47-9bc8-60a1928812dc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:43.438314 [ 7099 ] {fef4b85c-eaca-4152-910d-64a433e52482} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36858) (comment: 02124_buffer_with_type_map_long.sh) (in query: SELECT * FROM t_buffer_map), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:43.439308 [ 7099 ] {fef4b85c-eaca-4152-910d-64a433e52482} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:43.826225 [ 7238 ] {5f3e867c-e44c-4705-8334-8abbbf70077c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59008) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:43.841769 [ 7238 ] {5f3e867c-e44c-4705-8334-8abbbf70077c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:44.651911 [ 7091 ] {ce029103-5acb-487f-8a7e-9c3e13525066} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=903). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36720) (comment: 02124_buffer_with_type_map_long.sh) (in query: INSERT INTO t_buffer_map SELECT (range(10), range(10)), (range(10), range(10)) from numbers(100)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:44.653913 [ 7091 ] {ce029103-5acb-487f-8a7e-9c3e13525066} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=903). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:45.495610 [ 7099 ] {a22229ec-68df-4188-91d8-00d27f4a672e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59026) (comment: 00143_transform_non_const_default.sql) (in query: SELECT transform(SearchEngineID, [2, 3], ['Яндекс', 'Google'], PageCharset) AS title, count() AS c FROM test.hits WHERE SearchEngineID != 0 GROUP BY title HAVING c > 0 ORDER BY c DESC LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978fdbe
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:09:46.332823 [ 7099 ] {a22229ec-68df-4188-91d8-00d27f4a672e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978fdbe
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:09:46.332858 [ 7089 ] {9fc71185-c7df-40c3-8b7f-cd6fa1ac60fb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36738) (comment: 02124_buffer_with_type_map_long.sh) (in query: SELECT * FROM t_buffer_map), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:46.333948 [ 7089 ] {9fc71185-c7df-40c3-8b7f-cd6fa1ac60fb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=888). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:46.332885 [ 6987 ] {bbad28ec-ed13-4a6f-9e5a-608fa7ea7a35} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=889). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59022) (comment: 02124_buffer_with_type_map_long.sh) (in query: INSERT INTO t_buffer_map SELECT (range(10), range(10)), (range(10), range(10)) from numbers(100)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:46.335743 [ 6987 ] {bbad28ec-ed13-4a6f-9e5a-608fa7ea7a35} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=889). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:09:46.367969 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=1): (while reading column data): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_1_21_4/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
19. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
20. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
25. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:637: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForOneColumn() const @ 0x000000003905f636
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:869: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b261
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
29. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
30. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:47.230155 [ 6987 ] {a994a2a4-755e-4e28-b45a-90725076929c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:52658) (comment: 00147_global_in_aggregate_function.sql) (in query: SELECT `__table1`.`UserID` AS `UserID` FROM `test`.`hits` AS `__table1`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:47.242777 [ 6987 ] {a994a2a4-755e-4e28-b45a-90725076929c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:47.586630 [ 3885 ] {b76d29c2-0675-45ea-a609-a573412433c4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=893). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36802) (comment: 02124_buffer_with_type_map_long.sh) (in query: SELECT * FROM t_buffer_map), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:47.587582 [ 3885 ] {b76d29c2-0675-45ea-a609-a573412433c4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=893). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:09:48.182626 [ 7089 ] {6cc47f5c-5a92-4e82-8eda-df128d0309cf} <Error> executeQuery: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59072) (comment: 00147_global_in_aggregate_function.sql) (in query: SELECT sum(UserID GLOBAL IN (SELECT UserID FROM remote('127.0.0.{1,2}', test.hits))) FROM remote('127.0.0.{1,2}', test.hits);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:09:48.184805 [ 7089 ] {6cc47f5c-5a92-4e82-8eda-df128d0309cf} <Error> TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:09:48.266632 [ 7591 ] {d3d292e3-462c-4edf-a248-5db88e9aea89} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:33876) (comment: 00147_global_in_aggregate_function.sql) (in query: SELECT `__table1`.`UserID` AS `UserID` FROM `test`.`hits` AS `__table1`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:48.363898 [ 7591 ] {d3d292e3-462c-4edf-a248-5db88e9aea89} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:48.364666 [ 7591 ] {d3d292e3-462c-4edf-a248-5db88e9aea89} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:09:50.020997 [ 3885 ] {8a162a3a-944f-4820-9338-00501bfecacd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59088) (comment: 00156_max_execution_speed_sample_merge.sql) (in query: SELECT count() FROM test.hits SAMPLE 1 / 2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:09:50.191885 [ 3885 ] {8a162a3a-944f-4820-9338-00501bfecacd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:09:50.869788 [ 3885 ] {27e93be0-1652-48f2-b023-5ce5a7eb496b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=36, jobs=36). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59108) (comment: 00076_system_columns_bytes.sql) (in query: -- NOTE: database = currentDatabase() is not mandatory
 SELECT sum(data_compressed_bytes) > 0, sum(data_uncompressed_bytes) > 0, sum(marks_bytes) > 0 FROM system.columns;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:50.876442 [ 3885 ] {27e93be0-1652-48f2-b023-5ce5a7eb496b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=36, jobs=36). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:50.943580 [ 7091 ] {cbc69bf5-f895-42db-ae7d-4a0b74612da9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59102) (comment: 01221_system_settings.sql) (in query: with [ 'Seconds', 'Bool', 'Int64', 'String', 'Char', 'LogsLevel', 'URI', 'Float', 'UInt64', 'MaxThreads', 'Milliseconds', 'JoinStrictness', 'JoinAlgorithm', 'OverflowMode', 'TotalsMode', 'LoadBalancing', 'OverflowModeGroupBy', 'DateTimeInputFormat', 'DistributedProductMode' ] as types select hasAll(arrayDistinct(groupArray(type)), types) from system.settings;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:50.946983 [ 7091 ] {cbc69bf5-f895-42db-ae7d-4a0b74612da9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:51.234071 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_27_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationNullable::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c6ace1
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:51.553325 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:09:51.553442 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:09:52.135676 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/pmp/uwluyhaprsoguodmedxpmhitwgilj, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8a737e3d-f4be-4640-9ccf-9158d4eb50b3, upload has finished false.
2024.11.04 23:09:52.135779 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/pmp/uwluyhaprsoguodmedxpmhitwgilj, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8a737e3d-f4be-4640-9ccf-9158d4eb50b3, upload has finished false
2024.11.04 23:09:52.173289 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/pmp/uwluyhaprsoguodmedxpmhitwgilj, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8a737e3d-f4be-4640-9ccf-9158d4eb50b3, upload has finished false
2024.11.04 23:09:52.177428 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 35 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:54.040910 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {33f0dc8f-1542-43c0-a136-e415d11993b4::202411_1_31_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:56.508589 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:57.315784 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {33f0dc8f-1542-43c0-a136-e415d11993b4::202411_1_31_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:58.475965 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:58.476141 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_27_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=1): (while reading column version): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_1_21_4/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationNullable::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c6ace1
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
19. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
20. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
25. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:637: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForOneColumn() const @ 0x000000003905f636
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:869: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b261
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
29. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
30. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:58.490167 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/mrn/yyzrtutylbxzitqirjwbzkoshczuq, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 323d7cef-b993-42c7-9e0a-521f6c6943fd, upload has finished false.
2024.11.04 23:09:58.490334 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/mrn/yyzrtutylbxzitqirjwbzkoshczuq, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 323d7cef-b993-42c7-9e0a-521f6c6943fd, upload has finished false
2024.11.04 23:09:58.501932 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/mrn/yyzrtutylbxzitqirjwbzkoshczuq, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 323d7cef-b993-42c7-9e0a-521f6c6943fd, upload has finished false
2024.11.04 23:09:58.508476 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 49 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:59.006513 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:09:59.109882 [ 3885 ] {40cfe1b2-d839-4f7d-974d-9b16b905dbf3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59128) (comment: 02918_fuzzjson_table_function.sql) (in query: SELECT count() FROM 02918_table_str;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:59.112017 [ 3885 ] {40cfe1b2-d839-4f7d-974d-9b16b905dbf3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:09:59.987973 [ 3546 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:59.988079 [ 3547 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:59.988233 [ 3544 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:59.988416 [ 3549 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:59.988575 [ 3545 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:09:59.989243 [ 3557 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:00.689130 [ 3543 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
12. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
15. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
17. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
18. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:00.689239 [ 3551 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:00.689301 [ 3587 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3914: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x0000000038216241
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a61f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:00.701417 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_28_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:00.897358 [ 3885 ] {b475cffb-28b7-4cc3-a987-5114fb5e7902} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55336) (comment: 02950_dictionary_short_circuit.sql) (in query: SELECT dictGetOrDefault('hashed_dictionary', 'v2', 1, intDiv(1, id)) FROM dictionary_source_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:00.899703 [ 3885 ] {b475cffb-28b7-4cc3-a987-5114fb5e7902} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:02.365629 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:02.365633 [ 3885 ] {dcdf6b00-e049-4242-9257-4df9c44e0bac} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55346) (comment: 01513_defaults_on_defaults_no_column.sql) (in query: INSERT INTO defaults_on_defaults values ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:10:02.369311 [ 3885 ] {dcdf6b00-e049-4242-9257-4df9c44e0bac} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:10:03.380326 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_28_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:04.655346 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_29_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:07.233428 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_29_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:09.024555 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_29_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:09.446791 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_29_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:10.276768 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/xmx/xmkbvigzmlhkivluegltkfourzacy, total size 419430400, count 419430400, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f50bf112-9d87-488a-8b25-8fb2ea0f57cd, upload has finished false.
2024.11.04 23:10:10.276886 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/xmx/xmkbvigzmlhkivluegltkfourzacy, total size 419430400, count 419430400, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f50bf112-9d87-488a-8b25-8fb2ea0f57cd, upload has finished false
2024.11.04 23:10:10.330360 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/xmx/xmkbvigzmlhkivluegltkfourzacy, total size 419430400, count 419430400, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f50bf112-9d87-488a-8b25-8fb2ea0f57cd, upload has finished false
2024.11.04 23:10:10.335308 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_8_8_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 8 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:10.812952 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_29_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:11.983443 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_1_41_8}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:11.989698 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_29_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column event_date): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_1_21_4/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:12.199662 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 16 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:13.209035 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:13.710135 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:14.067009 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {a7e77ef5-c915-489e-89ce-773df4eb75ed::202411_1_31_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:14.798524 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:17.705174 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_30_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:19.117638 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_70_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:19.204718 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_30_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:19.786939 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_70_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:20.986092 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_30_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:22.419296 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/cga/axioceykeizeuyxacaijoqcmpdawi, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1c043c4d-71c3-423a-8619-db4b51b524af, upload has finished false.
2024.11.04 23:10:22.419517 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/cga/axioceykeizeuyxacaijoqcmpdawi, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1c043c4d-71c3-423a-8619-db4b51b524af, upload has finished false
2024.11.04 23:10:22.496909 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/cga/axioceykeizeuyxacaijoqcmpdawi, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1c043c4d-71c3-423a-8619-db4b51b524af, upload has finished false
2024.11.04 23:10:22.505738 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 77 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:23.391586 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:23.474968 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_22_26_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:23.516394 [ 7099 ] {753be408-9e1a-4028-aae4-56b5144d26b4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55216) (comment: 00159_parallel_formatting_json_and_friends_2.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactStringsEachRowWithNames), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:23.563788 [ 7099 ] {753be408-9e1a-4028-aae4-56b5144d26b4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:26.310597 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:26.311072 [ 7099 ] {8e18eb0d-128a-4b9a-849c-e8a096709093} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47640) (comment: 00159_parallel_formatting_json_and_friends_2.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactEachRowWithNamesAndTypes), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:10:27.919850 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:27.919878 [ 7099 ] {8e18eb0d-128a-4b9a-849c-e8a096709093} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:10:28.766947 [ 7099 ] {3f560808-e792-41fa-ad25-bd5fb7932904} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47648) (comment: 00159_parallel_formatting_json_and_friends_2.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactEachRowWithNamesAndTypes), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:28.783869 [ 7099 ] {3f560808-e792-41fa-ad25-bd5fb7932904} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:30.287563 [ 7099 ] {11d3de03-2dbd-416e-a456-fa74841018d9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47664) (comment: 00159_parallel_formatting_json_and_friends_2.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactStringsEachRowWithNamesAndTypes), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:30.345437 [ 7099 ] {11d3de03-2dbd-416e-a456-fa74841018d9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:31.296802 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_31_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:31.493373 [ 7099 ] {ecdfb6ae-1822-45fd-8359-833f47152f70} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47666) (comment: 00159_parallel_formatting_json_and_friends_2.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format JSONCompactStringsEachRowWithNamesAndTypes), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:10:31.545916 [ 7099 ] {ecdfb6ae-1822-45fd-8359-833f47152f70} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:10:32.511142 [ 7099 ] {c3557398-69e0-4270-aecc-7ca67eef8811} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47672) (comment: 00067_union_all.sql) (in query: SELECT * FROM ( SELECT UserID AS id, 1 AS event FROM remote('127.0.0.{1,2}', test, hits) ORDER BY id DESC LIMIT 10 UNION ALL SELECT FUniqID AS id, 2 AS event FROM remote('127.0.0.{1,2}', test, hits) ORDER BY id DESC LIMIT 10 ) ORDER BY id, event;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:32.645800 [ 7099 ] {c3557398-69e0-4270-aecc-7ca67eef8811} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:10:32.769468 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/akz/aezfoeoqwefnrcsfcfybohpuuqjuc, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1b8eec9d-2d0a-4cdc-bd3a-de560b2e96f5, upload has finished false.
2024.11.04 23:10:32.769657 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/akz/aezfoeoqwefnrcsfcfybohpuuqjuc, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1b8eec9d-2d0a-4cdc-bd3a-de560b2e96f5, upload has finished false
2024.11.04 23:10:32.810886 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/akz/aezfoeoqwefnrcsfcfybohpuuqjuc, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1b8eec9d-2d0a-4cdc-bd3a-de560b2e96f5, upload has finished false
2024.11.04 23:10:32.816448 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=41, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 69 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:35.958342 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:37.595894 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:37.596085 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_32_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:37.635731 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/fdz/rexyedxwlikqepagzbzxfndkbxdwc, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a96de998-fa78-4759-b527-6d1de5f8a20d, upload has finished false.
2024.11.04 23:10:37.635819 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/fdz/rexyedxwlikqepagzbzxfndkbxdwc, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a96de998-fa78-4759-b527-6d1de5f8a20d, upload has finished false
2024.11.04 23:10:37.666158 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/fdz/rexyedxwlikqepagzbzxfndkbxdwc, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a96de998-fa78-4759-b527-6d1de5f8a20d, upload has finished false
2024.11.04 23:10:37.711160 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 5 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:39.936558 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:40.006581 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 82 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:40.741259 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_33_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=8): (while reading column path): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_1_21_4/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:637: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForOneColumn() const @ 0x000000003905f636
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:869: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b261
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
29. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:42.607293 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:42.625683 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_33_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:44.592045 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:44.806277 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_33_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:45.291827 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/sct/kfvzrxneprefqyavwylhlkmqooodq, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 052ccb5f-eb70-47b2-876e-3af12c5e05e7, upload has finished false.
2024.11.04 23:10:45.291931 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/sct/kfvzrxneprefqyavwylhlkmqooodq, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 052ccb5f-eb70-47b2-876e-3af12c5e05e7, upload has finished false
2024.11.04 23:10:45.301629 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/sct/kfvzrxneprefqyavwylhlkmqooodq, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 052ccb5f-eb70-47b2-876e-3af12c5e05e7, upload has finished false
2024.11.04 23:10:45.305598 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 57 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:47.757921 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 85 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:48.813225 [ 7099 ] {c9ba90ef-9de6-4095-889c-d1874cfc2135} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=17): (while reading column RequestTry): (while reading from part /var/lib/clickhouse/disks/s3_disk/store/d3c/d3cd821f-b25e-4a50-855c-4cb59bd92d3a/201403_1_6_1/ in table test.hits_s3 (d3cd821f-b25e-4a50-855c-4cb59bd92d3a) located on disk s3_disk of type s3, from mark 75 with max_rows_to_read = 196608): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55778) (comment: 00180_no_seek_avoiding_when_reading_from_cache.sh) (in query: INSERT INTO hits_s3_sampled SELECT * FROM test.hits_s3 SAMPLE 0.01), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x0000000039756a72
18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000003977b0fa
19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x0000000039773549
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:164: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x00000000397939a3
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000003b72e014
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:139: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e3b1
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
29. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:10:48.840288 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_33_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:49.113633 [ 7099 ] {c9ba90ef-9de6-4095-889c-d1874cfc2135} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=17): (while reading column RequestTry): (while reading from part /var/lib/clickhouse/disks/s3_disk/store/d3c/d3cd821f-b25e-4a50-855c-4cb59bd92d3a/201403_1_6_1/ in table test.hits_s3 (d3cd821f-b25e-4a50-855c-4cb59bd92d3a) located on disk s3_disk of type s3, from mark 75 with max_rows_to_read = 196608): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x0000000039756a72
18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000003977b0fa
19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x0000000039773549
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:164: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x00000000397939a3
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000003b72e014
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:139: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e3b1
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
29. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:10:50.421524 [ 3550 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:10:50.421607 [ 3568 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:10:50.421668 [ 3567 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:10:50.421784 [ 3562 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:10:50.421887 [ 3580 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:10:50.422293 [ 3577 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:50.422511 [ 3575 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
12. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
15. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
17. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
18. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:50.422548 [ 3576 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:50.859630 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_34_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column thread_id): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_22_26_1/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:637: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForOneColumn() const @ 0x000000003905f636
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:869: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b261
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
29. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:52.667014 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_34_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:53.069129 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/sbn/kwhudzaavwkunnucnieaaqbbfimsj, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 482e0e32-5ecb-4dfb-9004-def9ad618403, upload has finished false.
2024.11.04 23:10:53.069276 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/sbn/kwhudzaavwkunnucnieaaqbbfimsj, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 482e0e32-5ecb-4dfb-9004-def9ad618403, upload has finished false
2024.11.04 23:10:53.079291 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/sbn/kwhudzaavwkunnucnieaaqbbfimsj, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 482e0e32-5ecb-4dfb-9004-def9ad618403, upload has finished false
2024.11.04 23:10:53.147904 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 8 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:54.015639 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {54259a93-f7ca-4ac1-8b7b-0c01fa06532c::202411_1_46_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:54.276962 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_34_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:56.561777 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:57.142692 [ 7099 ] {caa50cde-6eb9-4bf3-aab1-81fd05efbf84} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33356) (comment: 00089_position_functions_with_non_constant_arg.sql) (in query: SELECT count() FROM test.hits WHERE position(URL, 'metrika') != position(URL, materialize('metrika'));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:10:57.144411 [ 7099 ] {caa50cde-6eb9-4bf3-aab1-81fd05efbf84} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:10:58.323024 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_34_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:10:58.971292 [ 7099 ] {e7a460b6-003c-4d2f-a2f1-b23f2f8d8b59} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33368) (comment: 00139_like.sql) (in query: /* Note that queries are written as the user doesn't really understand that the symbol _ has special meaning in LIKE pattern. */
 SELECT count() FROM test.hits WHERE URL LIKE '%/avtomobili_s_probegom/_%__%__%__%';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:10:58.973094 [ 7099 ] {e7a460b6-003c-4d2f-a2f1-b23f2f8d8b59} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:10:59.387652 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_35_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:01.319522 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:01.319672 [ 7099 ] {7cc3c413-b0b6-4beb-984e-76ea29dbc7bc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33380) (comment: 00093_prewhere_array_join.sql) (in query: SELECT arrayJoin([SearchEngineID]) AS search_engine, URL FROM test.hits WHERE SearchEngineID != 0 AND search_engine != 0 FORMAT Null;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:01.334229 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/oum/fhojncvludzlehrbavtxpalvwxfxt, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b2f28fc3-f9b6-433a-876e-ad1c48b1224f, upload has finished false.
2024.11.04 23:11:01.334332 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/oum/fhojncvludzlehrbavtxpalvwxfxt, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b2f28fc3-f9b6-433a-876e-ad1c48b1224f, upload has finished false
2024.11.04 23:11:01.393242 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/oum/fhojncvludzlehrbavtxpalvwxfxt, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b2f28fc3-f9b6-433a-876e-ad1c48b1224f, upload has finished false
2024.11.04 23:11:03.215845 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:03.215921 [ 7099 ] {7cc3c413-b0b6-4beb-984e-76ea29dbc7bc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:03.216119 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 17 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:03.216200 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_35_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:03.734028 [ 7099 ] {11b11a41-c01b-409e-a1bc-7b3ca2e12845} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60648) (comment: 00051_min_max_array.sql) (in query: SELECT CounterID, count(), max(GoalsReached), min(GoalsReached), minIf(GoalsReached, notEmpty(GoalsReached)) FROM test.hits GROUP BY CounterID ORDER BY count() DESC LIMIT 20;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:03.735990 [ 7099 ] {11b11a41-c01b-409e-a1bc-7b3ca2e12845} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:05.659558 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_37_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:05.803551 [ 7099 ] {4b3fb3fc-8f0a-4dd1-a38d-aba7a04995ca} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=45, jobs=45). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60652) (comment: 00082_quantiles.sql) (in query: SELECT CounterID AS k, quantilesExact(0.1, 0.5, 0.9, 0.99, 0.999)(ResolutionWidth) FROM test.hits GROUP BY k ORDER BY count() DESC, CounterID LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:05.849915 [ 7099 ] {4b3fb3fc-8f0a-4dd1-a38d-aba7a04995ca} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=45, jobs=45). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:06.521025 [ 3885 ] {c0acf200-31ba-4247-94ab-f49f441218d7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60656) (comment: 00038_uniq_state_merge2.sql) (in query: SELECT topLevelDomain(concat('http://', k)) AS tld, sum(u) AS u, uniqMerge(us) AS us FROM (SELECT domain(URL) AS k, uniq(UserID) AS u, uniqState(UserID) AS us FROM test.hits GROUP BY k) GROUP BY tld ORDER BY u DESC, tld ASC LIMIT 100 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:06.539979 [ 3885 ] {c0acf200-31ba-4247-94ab-f49f441218d7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:06.889808 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_37_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column event_date): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_22_26_1/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:07.250449 [ 3885 ] {eaa9f6f0-6ae8-43fe-b920-9e0915062ff2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60676) (comment: 00009_uniq_distributed.sql) (in query: SELECT uniq(UserID), uniqIf(UserID, CounterID = 800784), uniqIf(FUniqID, RegionID = 213) FROM remote('127.0.0.{1,2}', test, hits) ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:07.253836 [ 3885 ] {eaa9f6f0-6ae8-43fe-b920-9e0915062ff2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:08.704162 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:08.704298 [ 7099 ] {5687fd33-7184-4edc-be0c-bf943f1e6b68} <Error> executeQuery: Code: 10. DB::Exception: Column `json_str` not found in table test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2). (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60664) (comment: 01710_projection_query_plan_optimization_misc.sql) (in query: select * from merge('', 't');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageSnapshot.cpp:214: DB::StorageSnapshot::getSampleBlockForColumns(std::vector<String, std::allocator<String>> const&) const @ 0x000000003865fde4
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::ReadFromMergeTree(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>>, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const> const&, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<Poco::Logger>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) @ 0x000000003b69f2ba
5. ./contrib/llvm-project/libcxx/include/vector:434: std::__unique_if<DB::ReadFromMergeTree>::__unique_single std::make_unique[abi:v15000]<DB::ReadFromMergeTree, std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&>(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&) @ 0x000000003946b1ef
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:921: DB::MergeTreeDataSelectExecutor::readFromParts(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) const @ 0x000000003942bebf
7. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:140: DB::MergeTreeDataSelectExecutor::read(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, bool) const @ 0x000000003942b247
8. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::readLocalImpl(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long) @ 0x0000000038274499
9. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000038270fe1
10. ./src/Processors/QueryPlan/QueryPlan.h:54: DB::ReadFromMerge::createPlanForTable(std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, DB::QueryProcessingStage::Enum, unsigned long, std::tuple<String, std::shared_ptr<DB::IStorage>, std::shared_ptr<DB::RWLockImpl::LockHolderImpl>, String> const&, std::vector<String, std::allocator<String>>&&, std::optional<DB::ReadFromMerge::RowPolicyData> const&, std::shared_ptr<DB::Context>, unsigned long) const @ 0x0000000037f6fb8f
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: DB::ReadFromMerge::createChildrenPlans(DB::SelectQueryInfo&) const @ 0x0000000037f5bbeb
12. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadFromMerge::filterTablesAndCreateChildrenPlans() @ 0x0000000037f4ad18
13. ./build_docker/./src/Storages/StorageMerge.cpp:1576: DB::ReadFromMerge::applyFilters(DB::ActionDAGNodes) @ 0x0000000037f87a1d
14. ./src/Processors/QueryPlan/SourceStepWithFilter.h:0: DB::QueryPlanOptimizations::optimizePrimaryKeyCondition(std::vector<DB::QueryPlanOptimizations::Frame, std::allocator<DB::QueryPlanOptimizations::Frame>> const&) @ 0x000000003b8f5b49
15. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:0: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list<DB::QueryPlan::Node, std::allocator<DB::QueryPlan::Node>>&) @ 0x000000003b8ab2c2
16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000003b66ac55
17. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:0: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b667910
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
23. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
27. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
28. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:11:08.704342 [ 3885 ] {39da28ac-6045-4179-b020-3f13112d47d3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=42, jobs=42). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60680) (comment: 00078_group_by_arrays.sql) (in query: SELECT GoalsReached AS k, count() AS c FROM test.hits GROUP BY k ORDER BY c DESC LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:08.706676 [ 7099 ] {5687fd33-7184-4edc-be0c-bf943f1e6b68} <Error> TCPHandler: Code: 10. DB::Exception: Column `json_str` not found in table test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2). (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageSnapshot.cpp:214: DB::StorageSnapshot::getSampleBlockForColumns(std::vector<String, std::allocator<String>> const&) const @ 0x000000003865fde4
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::ReadFromMergeTree(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>>, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const> const&, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<Poco::Logger>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) @ 0x000000003b69f2ba
5. ./contrib/llvm-project/libcxx/include/vector:434: std::__unique_if<DB::ReadFromMergeTree>::__unique_single std::make_unique[abi:v15000]<DB::ReadFromMergeTree, std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&>(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&) @ 0x000000003946b1ef
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:921: DB::MergeTreeDataSelectExecutor::readFromParts(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) const @ 0x000000003942bebf
7. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:140: DB::MergeTreeDataSelectExecutor::read(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, bool) const @ 0x000000003942b247
8. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::readLocalImpl(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long) @ 0x0000000038274499
9. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000038270fe1
10. ./src/Processors/QueryPlan/QueryPlan.h:54: DB::ReadFromMerge::createPlanForTable(std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, DB::QueryProcessingStage::Enum, unsigned long, std::tuple<String, std::shared_ptr<DB::IStorage>, std::shared_ptr<DB::RWLockImpl::LockHolderImpl>, String> const&, std::vector<String, std::allocator<String>>&&, std::optional<DB::ReadFromMerge::RowPolicyData> const&, std::shared_ptr<DB::Context>, unsigned long) const @ 0x0000000037f6fb8f
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: DB::ReadFromMerge::createChildrenPlans(DB::SelectQueryInfo&) const @ 0x0000000037f5bbeb
12. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadFromMerge::filterTablesAndCreateChildrenPlans() @ 0x0000000037f4ad18
13. ./build_docker/./src/Storages/StorageMerge.cpp:1576: DB::ReadFromMerge::applyFilters(DB::ActionDAGNodes) @ 0x0000000037f87a1d
14. ./src/Processors/QueryPlan/SourceStepWithFilter.h:0: DB::QueryPlanOptimizations::optimizePrimaryKeyCondition(std::vector<DB::QueryPlanOptimizations::Frame, std::allocator<DB::QueryPlanOptimizations::Frame>> const&) @ 0x000000003b8f5b49
15. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:0: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list<DB::QueryPlan::Node, std::allocator<DB::QueryPlan::Node>>&) @ 0x000000003b8ab2c2
16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000003b66ac55
17. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:0: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b667910
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
23. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
27. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
28. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:11:08.725883 [ 3286 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:08.725982 [ 3282 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:08.726057 [ 3281 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:08.726153 [ 3287 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:08.726258 [ 3285 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:08.727240 [ 3294 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:08.727350 [ 3277 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:08.736112 [ 3284 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:08.736090 [ 3283 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:08.736197 [ 3291 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:08.745928 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_37_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=1): (while reading column event_time): (while reading from part /var/lib/clickhouse/disks/s3/store/386/386c9b4d-67f8-4db6-a90a-ff6424edf355/202411_22_26_1/ in table system.zookeeper_log (386c9b4d-67f8-4db6-a90a-ff6424edf355) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:08.847151 [ 3885 ] {39da28ac-6045-4179-b020-3f13112d47d3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=42, jobs=42). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:10.757532 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/xsm/jtiyutjgmqwprsydqvczfjpyyhavl, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ec963c18-a73d-41b4-bef5-8f5c02af3f29, upload has finished false.
2024.11.04 23:11:10.757665 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/xsm/jtiyutjgmqwprsydqvczfjpyyhavl, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ec963c18-a73d-41b4-bef5-8f5c02af3f29, upload has finished false
2024.11.04 23:11:10.788504 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/xsm/jtiyutjgmqwprsydqvczfjpyyhavl, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ec963c18-a73d-41b4-bef5-8f5c02af3f29, upload has finished false
2024.11.04 23:11:11.399184 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 58 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:11.399339 [ 3949 ] {b939231b-b952-47a6-92fe-8642594fb5e3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=3): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60684) (comment: 00010_quantiles_segfault.sql) (in query: SELECT URL AS `ym:ah:URL`, sum((NOT DontCountHits AND NOT Refresh)), quantilesTimingIf(0.1, 0.5, 0.9)((DOMCompleteTiming + LoadEventEndTiming), DOMCompleteTiming != -1 AND LoadEventEndTiming != -1) as t FROM remote('127.0.0.{1,2}', test, hits) WHERE (CounterID = 800784) AND (((DontCountHits = 0) OR (IsNotBounce = 1)) AND (URL != '')) GROUP BY `ym:ah:URL` WITH TOTALS HAVING (sum((NOT DontCountHits AND NOT Refresh)) > 0) AND (count() > 0) ORDER BY sum((NOT DontCountHits AND NOT Refresh)) DESC, URL LIMIT 0, 1 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003b75a78e
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003b73fe83
8. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::PrefetchedReaders::PrefetchedReaders(DB::MergeTreeReadTask::Readers, Priority, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b73ec1d
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::MergeTreePrefetchedReadPool::PrefetchedReaders>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreePrefetchedReadPool::PrefetchedReaders, DB::MergeTreeReadTask::Readers, Priority&, DB::MergeTreePrefetchedReadPool&>(DB::MergeTreeReadTask::Readers&&, Priority&, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b75d6fc
10. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:167: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b750566
11. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
12. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
13. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
14. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
15. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
16. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
17. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
19. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
20. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
21. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:12.407659 [ 3949 ] {b939231b-b952-47a6-92fe-8642594fb5e3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=3): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003b75a78e
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003b73fe83
8. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::PrefetchedReaders::PrefetchedReaders(DB::MergeTreeReadTask::Readers, Priority, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b73ec1d
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::MergeTreePrefetchedReadPool::PrefetchedReaders>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreePrefetchedReadPool::PrefetchedReaders, DB::MergeTreeReadTask::Readers, Priority&, DB::MergeTreePrefetchedReadPool&>(DB::MergeTreeReadTask::Readers&&, Priority&, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b75d6fc
10. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:167: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b750566
11. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
12. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
13. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
14. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
15. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
16. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
17. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
19. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
20. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
21. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:12.407678 [ 7238 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:59238). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:12.455049 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:14.268559 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:14.269253 [ 3885 ] {45ad4b5b-ad3e-42a2-b9ae-643f60e96c25} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=34, jobs=34). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60714) (comment: 00077_log_tinylog_stripelog.sql) (in query: CHECK TABLE test.hits_log;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:14.274958 [ 3885 ] {45ad4b5b-ad3e-42a2-b9ae-643f60e96c25} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=34, jobs=34). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:14.310855 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_1_37_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:15.060029 [ 3949 ] {dbcc1138-c087-4864-9ee1-9fafb7d33c6d} <Error> executeQuery: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51016) (comment: 01397_in_bad_arguments.sql) (in query: select in((1, 1, 1, 1));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.061496 [ 3949 ] {dbcc1138-c087-4864-9ee1-9fafb7d33c6d} <Error> TCPHandler: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.075056 [ 3949 ] {838134af-7eb2-4e4f-9634-53c2e78a4227} <Error> executeQuery: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51016) (comment: 01397_in_bad_arguments.sql) (in query: select in(1);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.094763 [ 3949 ] {838134af-7eb2-4e4f-9634-53c2e78a4227} <Error> TCPHandler: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.106672 [ 3949 ] {895e0716-043d-4adb-83db-5c8574d58996} <Error> executeQuery: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51016) (comment: 01397_in_bad_arguments.sql) (in query: select in();), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.107655 [ 3949 ] {895e0716-043d-4adb-83db-5c8574d58996} <Error> TCPHandler: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.118980 [ 3949 ] {4497456d-34e8-4aec-add9-b7eb43d66643} <Error> executeQuery: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51016) (comment: 01397_in_bad_arguments.sql) (in query: select in(1, 2, 3);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.119895 [ 3949 ] {4497456d-34e8-4aec-add9-b7eb43d66643} <Error> TCPHandler: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x0000000007cc36b5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5594: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034823314
4. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.324994 [ 3885 ] {704844a9-4b8b-4532-ac79-c07e2f3b5ae4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51020) (comment: 00044_any_left_join_string.sql) (in query: SELECT domain, hits, visits FROM ( SELECT domain(URL) AS domain, count() AS hits FROM test.hits GROUP BY domain ) ANY LEFT JOIN ( SELECT domain(StartURL) AS domain, sum(Sign) AS visits FROM test.visits GROUP BY domain ) USING domain ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.349715 [ 3885 ] {704844a9-4b8b-4532-ac79-c07e2f3b5ae4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.424662 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_38_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:15.641417 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:15.861533 [ 3885 ] {ea83e1af-c980-4ae4-9975-cfc4e0673ecd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51036) (comment: 02991_count_rewrite_analyzer.sql) (in query: SELECT toTypeName(count(toNullable('a') IN toNullable('a'))) AS x;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:15.865227 [ 3885 ] {ea83e1af-c980-4ae4-9975-cfc4e0673ecd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:16.806848 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_38_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:16.871981 [ 3949 ] {c4ed9251-aa5e-4d1b-8313-df9c8c0b0148} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51052) (comment: 01378_alter_rename_with_ttl_zookeeper.sql) (in query: SELECT count() FROM table_rename_with_ttl;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:16.879776 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/uxl/iyzonnrjuzgscxxltcmkksgtwhqum, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b537d816-3950-4cb6-b76b-f76b0fe90edc, upload has finished false.
2024.11.04 23:11:16.879794 [ 3949 ] {c4ed9251-aa5e-4d1b-8313-df9c8c0b0148} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:16.879947 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/uxl/iyzonnrjuzgscxxltcmkksgtwhqum, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b537d816-3950-4cb6-b76b-f76b0fe90edc, upload has finished false
2024.11.04 23:11:16.965252 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/uxl/iyzonnrjuzgscxxltcmkksgtwhqum, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b537d816-3950-4cb6-b76b-f76b0fe90edc, upload has finished false
2024.11.04 23:11:16.969851 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 13 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:18.641306 [ 3949 ] {2e9aae22-077a-4374-bef6-6436f9ffb7fa} <Error> executeQuery: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasTokenCaseInsensitive('K(G', ''). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51064) (comment: 02816_has_token_empty.sql) (in query: SELECT hasTokenCaseInsensitive('K(G', '');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b1fc53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b1ca9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:18.641556 [ 3885 ] {0d7ad729-7567-4d10-96a8-7d911f20c956} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51046) (comment: 00055_index_and_not.sql) (in query: SELECT count() FROM test.hits WHERE NOT (EventDate >= toDate('2015-01-01') AND EventDate < toDate('2015-02-01')) ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:18.642376 [ 3949 ] {2e9aae22-077a-4374-bef6-6436f9ffb7fa} <Error> TCPHandler: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasTokenCaseInsensitive('K(G', ''). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b1fc53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b1ca9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:18.650748 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_38_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:18.656291 [ 3949 ] {7a80a451-7ce6-4a6b-aeea-d5911a17ee35} <Error> executeQuery: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasTokenCaseInsensitive('Hello', ''). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51064) (comment: 02816_has_token_empty.sql) (in query: SELECT hasTokenCaseInsensitive('Hello', '');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b1fc53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b1ca9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:18.657028 [ 3949 ] {7a80a451-7ce6-4a6b-aeea-d5911a17ee35} <Error> TCPHandler: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasTokenCaseInsensitive('Hello', ''). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b1fc53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b1ca9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:18.666211 [ 3949 ] {cedfc4aa-b4f2-48a3-8fcc-ab1be95a8173} <Error> executeQuery: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasTokenCaseInsensitive('', ''). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51064) (comment: 02816_has_token_empty.sql) (in query: SELECT hasTokenCaseInsensitive('', '');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b1fc53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b1ca9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:18.666769 [ 3949 ] {cedfc4aa-b4f2-48a3-8fcc-ab1be95a8173} <Error> TCPHandler: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasTokenCaseInsensitive('', ''). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b1fc53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasTokenCaseInsensitive, DB::VolnitskyBase<false, true, DB::impl::StringSearcher<false, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b1ca9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:18.677916 [ 3885 ] {0d7ad729-7567-4d10-96a8-7d911f20c956} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:19.870946 [ 3949 ] {65b11bd7-d9a8-4431-bb31-afdba55897f8} <Error> executeQuery: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasToken('Hello', ''). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51064) (comment: 02816_has_token_empty.sql) (in query: SELECT hasToken('Hello', '');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b13f53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b10d9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:19.871945 [ 3949 ] {65b11bd7-d9a8-4431-bb31-afdba55897f8} <Error> TCPHandler: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasToken('Hello', ''). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b13f53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b10d9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:19.877929 [ 3157 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:19.877985 [ 3163 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:19.878055 [ 3146 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:19.878135 [ 3162 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:19.878561 [ 3166 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:19.878659 [ 3169 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:19.882514 [ 3160 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:19.906980 [ 3949 ] {6725bcff-7102-45e4-96e5-c920f5b149ab} <Error> executeQuery: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasToken('', ''). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51064) (comment: 02816_has_token_empty.sql) (in query: SELECT hasToken('', '');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b13f53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b10d9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:19.907880 [ 3949 ] {6725bcff-7102-45e4-96e5-c920f5b149ab} <Error> TCPHandler: Code: 36. DB::Exception: Needle cannot be empty, because empty string isn't a token: In scope SELECT hasToken('', ''). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>::vectorConstant(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, String const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::ColumnVector<char8_t>*) @ 0x0000000011b13f53
4. DB::FunctionsStringSearch<DB::HasTokenImpl<DB::NameHasToken, DB::VolnitskyBase<true, true, DB::impl::StringSearcher<true, true>>, false>, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000011b10d9f
5. DB::IFunction::executeImplDryRun(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000007c62ee2
6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede718
7. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c1ba
8. ./build_docker/./src/Functions/IFunction.cpp:195: DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4ae40
9. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c096
10. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034819cb6
13. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
18. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:21.377226 [ 8002 ] {9a0c19bb-07a4-4337-8b00-59c577617338} <Error> executeQuery: Code: 16. DB::Exception: Column b does not exist in the materialized view's inner table. (NO_SUCH_COLUMN_IN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51080) (comment: 03022_alter_materialized_view_query_has_inner_table.sql) (in query: ALTER TABLE mv MODIFY QUERY SELECT a, b FROM src_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageMaterializedView.cpp:79: DB::StorageMaterializedView::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x0000000037eedc2c
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:11:21.378894 [ 8002 ] {9a0c19bb-07a4-4337-8b00-59c577617338} <Error> TCPHandler: Code: 16. DB::Exception: Column b does not exist in the materialized view's inner table. (NO_SUCH_COLUMN_IN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageMaterializedView.cpp:79: DB::StorageMaterializedView::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x0000000037eedc2c
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:11:22.387468 [ 3949 ] {2b858ab2-4854-49fb-a49c-7f510b1cdfd4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51100) (comment: 01147_partial_merge_full_join.sql) (in query: CREATE TABLE t1 (`x` UInt32, `y` UInt64) ENGINE = MergeTree ORDER BY (x, y) SETTINGS index_granularity = 18305, min_bytes_for_wide_part = 377997399, ratio_of_defaults_for_sparse_serialization = 0.7129498720169067, replace_long_file_name_to_hash = true, max_file_name_length = 7, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 394514900, compact_parts_max_granules_to_buffer = 193, compact_parts_merge_max_bytes_to_prefetch_part = 15261032, merge_max_block_size = 1860, old_parts_lifetime = 30., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 10737418240, index_granularity_bytes = 1945692, concurrent_part_removal_threshold = 100, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 65367, primary_key_compress_block_size = 95633), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::finalizeImpl() @ 0x0000000032472254
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:863: ? @ 0x00000000324714f9
14. DB::WriteBuffer::finalize() @ 0x0000000007c88f2e
15. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:307: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000390bcec2
16. ./build_docker/./src/Storages/StorageMergeTree.cpp:120: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04626
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
19. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
20. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
21. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
22. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
23. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:22.388686 [ 3949 ] {2b858ab2-4854-49fb-a49c-7f510b1cdfd4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::finalizeImpl() @ 0x0000000032472254
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:863: ? @ 0x00000000324714f9
14. DB::WriteBuffer::finalize() @ 0x0000000007c88f2e
15. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:307: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000390bcec2
16. ./build_docker/./src/Storages/StorageMergeTree.cpp:120: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04626
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
19. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
20. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
21. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
22. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
23. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
24. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
31. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e

2024.11.04 23:11:22.589539 [ 7099 ] {1de2d5b8-d822-4bde-b361-3f84699b84cd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51088) (comment: 00050_min_max.sql) (in query: SELECT CounterID, min(WatchID), max(WatchID) FROM test.hits GROUP BY CounterID ORDER BY count() DESC LIMIT 20;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:22.648293 [ 7099 ] {1de2d5b8-d822-4bde-b361-3f84699b84cd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:23.156761 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_38_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:23.592504 [ 7238 ] {c20e51cf-83ad-4f88-bcb6-7e8c583bf33a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40952) (comment: 00016_any_if_distributed_cond_always_false.sql) (in query: SELECT anyIf(`__table1`.`SearchPhrase`, `__table1`.`CounterID` = -1) AS `anyIf(SearchPhrase, equals(CounterID, -1))` FROM `test`.`hits` AS `__table1`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:23.597657 [ 7238 ] {c20e51cf-83ad-4f88-bcb6-7e8c583bf33a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:23.669558 [ 3949 ] {882ea066-66b2-4c08-acb3-e8dc9933aa75} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58258) (comment: 00016_any_if_distributed_cond_always_false.sql) (in query: SELECT anyIf(`__table1`.`SearchPhrase`, `__table1`.`CounterID` = -1) AS `anyIf(SearchPhrase, equals(CounterID, -1))` FROM `test`.`hits` AS `__table1`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:23.671579 [ 3949 ] {882ea066-66b2-4c08-acb3-e8dc9933aa75} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:23.675057 [ 7099 ] {c194f199-7783-44f2-b96c-ad5f9718ed47} <Error> executeQuery: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51108) (comment: 00016_any_if_distributed_cond_always_false.sql) (in query: SELECT anyIf(SearchPhrase, CounterID = -1) FROM remote('127.0.0.{1,2}:9000', test, hits) ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:11:23.681748 [ 7099 ] {c194f199-7783-44f2-b96c-ad5f9718ed47} <Error> TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:11:23.687193 [ 3949 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:58258). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:25.379804 [ 8002 ] {89047182-4af1-4bf3-8344-aa9b44db4d0b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=925). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:58264) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:11:25.395564 [ 3760 ] {} <Error> void DB::SystemLog<DB::QueryViewsLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryViewsLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryViewsLogElement>::flushImpl(std::vector<DB::QueryViewsLogElement, std::allocator<DB::QueryViewsLogElement>> const&, unsigned long) @ 0x000000003603648a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryViewsLogElement>::savingThreadFunction() @ 0x0000000036032be8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryViewsLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryViewsLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f435687
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43555f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:26.370253 [ 8002 ] {89047182-4af1-4bf3-8344-aa9b44db4d0b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=925). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:11:26.370314 [ 8079 ] {b43b87ec-2749-4911-adb4-186efdd30af2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column SendTiming): (while reading from part /var/lib/clickhouse/disks/s3_disk/store/cf2/cf277b48-304f-4234-856c-a75be6cc7c53/201403_1_6_1/ in table test.hits (cf277b48-304f-4234-856c-a75be6cc7c53) located on disk s3_disk of type s3, from mark 157 with max_rows_to_read = 16384): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45650) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x0000000039756a72
18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000003977a3f8
19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x0000000039773549
20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x0000000039773231
21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:164: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x00000000397939a3
22. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000003b78ea54
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:139: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e3b1
24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
25. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
26. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
28. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f

2024.11.04 23:11:26.370263 [ 8082 ] {8e2c5c5d-2f1d-4d0a-b78e-6484edea97a4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:56640) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:26.370417 [ 8085 ] {e00f8f95-e4d2-4861-969f-3724d1348eff} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:53082) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:26.370320 [ 8078 ] {822a014c-19fa-48b2-b0c8-16c6e368a49c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:40968) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:11:26.370489 [ 8080 ] {c63bf47a-4f33-4413-ba5a-9cbb25dfd5bd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:46360) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:11:26.400496 [ 8082 ] {8e2c5c5d-2f1d-4d0a-b78e-6484edea97a4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:26.403372 [ 8085 ] {e00f8f95-e4d2-4861-969f-3724d1348eff} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:26.411486 [ 8079 ] {b43b87ec-2749-4911-adb4-186efdd30af2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column SendTiming): (while reading from part /var/lib/clickhouse/disks/s3_disk/store/cf2/cf277b48-304f-4234-856c-a75be6cc7c53/201403_1_6_1/ in table test.hits (cf277b48-304f-4234-856c-a75be6cc7c53) located on disk s3_disk of type s3, from mark 157 with max_rows_to_read = 16384): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x0000000039756a72
18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000003977a3f8
19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x0000000039773549
20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x0000000039773231
21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:164: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x00000000397939a3
22. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000003b78ea54
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:139: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e3b1
24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
25. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
26. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
28. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f

2024.11.04 23:11:26.425328 [ 8078 ] {822a014c-19fa-48b2-b0c8-16c6e368a49c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:11:26.564723 [ 8080 ] {c63bf47a-4f33-4413-ba5a-9cbb25dfd5bd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:11:26.569860 [ 3949 ] {29e5c999-ede1-473c-9631-f6ca58b7d48c} <Error> executeQuery: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=925). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59150) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT sum(cityHash64(*)) FROM (SELECT CounterID, quantileTiming(0.5)(SendTiming), count() FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) WHERE SendTiming != -1 GROUP BY CounterID) SETTINGS max_block_size = 63169;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:11:26.670326 [ 8080 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46360). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:26.675066 [ 7238 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:44224). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:26.683802 [ 8078 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:40968). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:26.919351 [ 8083 ] {41843441-30de-470b-8c79-2df667a6e1e2} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:43136). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:43136) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9ea6
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:11:26.919397 [ 8081 ] {abe7f191-4a59-420e-9b25-2f6813efbb1b} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:45496). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45496) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9ea6
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:11:26.919376 [ 3949 ] {29e5c999-ede1-473c-9631-f6ca58b7d48c} <Error> TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=925). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:11:27.039781 [ 3704 ] {f841e245-6914-4a7d-9734-8d0743ba1f01::all_1_1_0_2} <Error> MutatePlainMergeTreeTask: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:27.615047 [ 8084 ] {f1f88c89-28d3-404a-82f3-371acfb02542} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:41378). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:41378) (comment: 00171_grouping_aggregated_transform_bug.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
5. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
6. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
7. ./src/AggregateFunctions/QuantileTiming.h:376: DB::detail::QuantileTimingLarge::serialize(DB::WriteBuffer&) const @ 0x0000000021a4596b
8. ./src/AggregateFunctions/QuantileTiming.h:703: DB::QuantileTiming<int>::serialize(DB::WriteBuffer&) const @ 0x0000000021a89401
9. ./src/AggregateFunctions/AggregateFunctionQuantile.h:243: DB::AggregateFunctionQuantile<int, DB::QuantileTiming<int>, DB::NameQuantileTiming, false, float, false, false>::serialize(char const*, DB::WriteBuffer&, std::optional<unsigned long>) const @ 0x0000000021a823ef
10. ./build_docker/./src/DataTypes/Serializations/SerializationAggregateFunction.cpp:71: DB::SerializationAggregateFunction::serializeBinaryBulk(DB::IColumn const&, DB::WriteBuffer&, unsigned long, unsigned long) const @ 0x0000000031bb90bf
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031ba0ede
12. ./build_docker/./src/Formats/NativeWriter.cpp:64: DB::NativeWriter::write(DB::Block const&) @ 0x000000003a5b1f01
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501e13
14. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9ea6
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:11:28.062461 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_1_51_10}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:28.062537 [ 8081 ] {abe7f191-4a59-420e-9b25-2f6813efbb1b} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:45496). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:28.062537 [ 8083 ] {41843441-30de-470b-8c79-2df667a6e1e2} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:43136). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:28.598512 [ 8002 ] {839a118d-c3d2-4356-a288-57a8034339ce} <Error> executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_2.txt' with part 'all_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59164) (comment: 01035_lc_empty_part_bug.sh) (in query: alter table lc_empty_part_bug delete where id < 100;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:11:28.598615 [ 8083 ] {41843441-30de-470b-8c79-2df667a6e1e2} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:43136). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9ea6
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:11:28.598714 [ 8084 ] {f1f88c89-28d3-404a-82f3-371acfb02542} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:41378). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:28.598783 [ 3704 ] {f841e245-6914-4a7d-9734-8d0743ba1f01::all_1_1_0_2} <Error> virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:28.598729 [ 8081 ] {abe7f191-4a59-420e-9b25-2f6813efbb1b} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:45496). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9ea6
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:11:28.599610 [ 8084 ] {f1f88c89-28d3-404a-82f3-371acfb02542} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:41378). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
5. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
6. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
7. ./src/AggregateFunctions/QuantileTiming.h:376: DB::detail::QuantileTimingLarge::serialize(DB::WriteBuffer&) const @ 0x0000000021a4596b
8. ./src/AggregateFunctions/QuantileTiming.h:703: DB::QuantileTiming<int>::serialize(DB::WriteBuffer&) const @ 0x0000000021a89401
9. ./src/AggregateFunctions/AggregateFunctionQuantile.h:243: DB::AggregateFunctionQuantile<int, DB::QuantileTiming<int>, DB::NameQuantileTiming, false, float, false, false>::serialize(char const*, DB::WriteBuffer&, std::optional<unsigned long>) const @ 0x0000000021a823ef
10. ./build_docker/./src/DataTypes/Serializations/SerializationAggregateFunction.cpp:71: DB::SerializationAggregateFunction::serializeBinaryBulk(DB::IColumn const&, DB::WriteBuffer&, unsigned long, unsigned long) const @ 0x0000000031bb90bf
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031ba0ede
12. ./build_docker/./src/Formats/NativeWriter.cpp:64: DB::NativeWriter::write(DB::Block const&) @ 0x000000003a5b1f01
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501e13
14. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9ea6
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:11:28.600212 [ 8081 ] {abe7f191-4a59-420e-9b25-2f6813efbb1b} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:11:28.600299 [ 8083 ] {41843441-30de-470b-8c79-2df667a6e1e2} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:11:28.601239 [ 8002 ] {839a118d-c3d2-4356-a288-57a8034339ce} <Error> TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_2.txt' with part 'all_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:11:28.611554 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/dcj/avghrumswaknnjlvaoimwhaojrovv, total size 385875968, count 385875968, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 085b5302-b303-4170-9838-01514b98bf5f, upload has finished false.
2024.11.04 23:11:28.611714 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/dcj/avghrumswaknnjlvaoimwhaojrovv, total size 385875968, count 385875968, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 085b5302-b303-4170-9838-01514b98bf5f, upload has finished false
2024.11.04 23:11:28.668157 [ 8084 ] {f1f88c89-28d3-404a-82f3-371acfb02542} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:11:28.729065 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/dcj/avghrumswaknnjlvaoimwhaojrovv, total size 385875968, count 385875968, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 085b5302-b303-4170-9838-01514b98bf5f, upload has finished false
2024.11.04 23:11:28.733233 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 49 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:30.811525 [ 7238 ] {f417f274-7a02-4d32-b973-8f540adaa3bb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59168) (comment: 01035_lc_empty_part_bug.sh) (in query: SELECT count() FROM lc_empty_part_bug WHERE not ignore(*);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:30.811571 [ 3948 ] {4a25b5e1-d48f-49d8-bd13-5caefc72651d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37854) (comment: 00159_parallel_formatting_http.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a,MobilePhoneModel as b,ClientIP6 as c FROM test.hits ORDER BY a,b,c LIMIT 1000000 Format TSV ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
19. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
20. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
21. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
23. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
24. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
25. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
26. ? @ 0x00007fe94d0adac3
27. ? @ 0x00007fe94d13f850

2024.11.04 23:11:30.811640 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:30.814591 [ 3948 ] {4a25b5e1-d48f-49d8-bd13-5caefc72651d} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
19. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
20. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
21. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
23. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
24. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
25. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
26. ? @ 0x00007fe94d0adac3
27. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:30.817279 [ 7238 ] {f417f274-7a02-4d32-b973-8f540adaa3bb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:30.933872 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_39_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:31.778208 [ 3948 ] {efa97470-6b5f-49ea-a057-b3632eb1ac14} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37868) (comment: 00159_parallel_formatting_http.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a,MobilePhoneModel as b,ClientIP6 as c FROM test.hits ORDER BY a,b,c LIMIT 1000000 Format TSV ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:11:32.135185 [ 3948 ] {efa97470-6b5f-49ea-a057-b3632eb1ac14} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:33.538808 [ 3948 ] {6873b67f-9bab-4559-a70b-0d3286838a58} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37876) (comment: 00159_parallel_formatting_http.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a,MobilePhoneModel as b,ClientIP6 as c FROM test.hits ORDER BY a,b,c LIMIT 1000000 Format CSV ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
19. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
20. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
21. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
23. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
24. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
25. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
26. ? @ 0x00007fe94d0adac3
27. ? @ 0x00007fe94d13f850

2024.11.04 23:11:33.544072 [ 3948 ] {6873b67f-9bab-4559-a70b-0d3286838a58} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
19. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
20. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
21. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
23. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
24. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
25. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
26. ? @ 0x00007fe94d0adac3
27. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:33.551378 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_40_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:34.563788 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/nrr/trqczbnvwkwjrhrsrgujqpwfufgqv, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6b94cff8-8533-44e4-9442-1cc83a743c8a, upload has finished false.
2024.11.04 23:11:34.563927 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/nrr/trqczbnvwkwjrhrsrgujqpwfufgqv, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6b94cff8-8533-44e4-9442-1cc83a743c8a, upload has finished false
2024.11.04 23:11:34.571546 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/nrr/trqczbnvwkwjrhrsrgujqpwfufgqv, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6b94cff8-8533-44e4-9442-1cc83a743c8a, upload has finished false
2024.11.04 23:11:34.578030 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 16 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:36.000919 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:36.472936 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_40_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:36.993660 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:40.498199 [ 8002 ] {47fb8976-4288-408a-b42f-8bbb2576f232} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60016) (comment: 00159_parallel_formatting_http.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a,MobilePhoneModel as b,ClientIP6 as c FROM test.hits ORDER BY a,b,c LIMIT 1000000 Format JSONCompactEachRow ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:11:40.589041 [ 8002 ] {47fb8976-4288-408a-b42f-8bbb2576f232} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:40.810137 [ 8002 ] {2fb89fc7-ca13-4875-9c2f-30f57cd7d470} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:60026) (comment: 00159_parallel_formatting_http.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a,MobilePhoneModel as b,ClientIP6 as c FROM test.hits ORDER BY a,b,c LIMIT 1000000 Format JSONCompactEachRow ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:11:40.889240 [ 8002 ] {2fb89fc7-ca13-4875-9c2f-30f57cd7d470} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:43.954820 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:44.605220 [ 7099 ] {ea14d697-ed79-4f1b-95fd-5aadb088bcf1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41476) (comment: 00063_loyalty_joins.sql) (in query: SELECT loyalty, count() FROM test.hits ANY LEFT JOIN ( SELECT UserID, sum(SearchEngineID = 2) AS yandex, sum(SearchEngineID = 3) AS google, toInt8(if(yandex > google, yandex / (yandex + google), -google / (yandex + google)) * 10) AS loyalty FROM test.hits WHERE (SearchEngineID = 2) OR (SearchEngineID = 3) GROUP BY UserID HAVING (yandex + google) > 10 ) USING UserID GROUP BY loyalty ORDER BY loyalty ASC;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:44.605288 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_41_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:44.607969 [ 7099 ] {ea14d697-ed79-4f1b-95fd-5aadb088bcf1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:44.723011 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_87_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:45.560584 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:47.915495 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_41_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:48.496900 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/kfg/bmmpnrfwjcqjagngkcbapithydrtm, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 74293edb-c0e6-449f-913c-3a51008e755b, upload has finished false.
2024.11.04 23:11:48.497054 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/kfg/bmmpnrfwjcqjagngkcbapithydrtm, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 74293edb-c0e6-449f-913c-3a51008e755b, upload has finished false
2024.11.04 23:11:48.551963 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/kfg/bmmpnrfwjcqjagngkcbapithydrtm, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 74293edb-c0e6-449f-913c-3a51008e755b, upload has finished false
2024.11.04 23:11:49.302204 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 9 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:49.302359 [ 7099 ] {0b7e575d-7774-4791-bad0-801501df635b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46154) (comment: 00049_max_string_if.sql) (in query: SELECT CounterID, count(), maxIf(SearchPhrase, notEmpty(SearchPhrase)) FROM test.hits GROUP BY CounterID ORDER BY count() DESC LIMIT 20;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:11:49.948782 [ 7099 ] {0b7e575d-7774-4791-bad0-801501df635b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:11:49.948800 [ 7238 ] {457e9889-0235-4b9a-b503-acd985afe924} <Error> executeQuery: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46156) (in query: select _path,_file from file('nonexist.txt', 'CSV', 'val1 char')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:11:49.948790 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:49.950177 [ 7238 ] {457e9889-0235-4b9a-b503-acd985afe924} <Error> TCPHandler: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:11:50.410320 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_41_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:50.559728 [ 7099 ] {08c9f7f8-c9e1-4220-a60a-63a44ca0b25a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46174) (comment: 00015_totals_and_no_aggregate_functions.sql) (in query: SELECT AdvEngineID FROM test.hits GROUP BY AdvEngineID WITH TOTALS ORDER BY AdvEngineID ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:50.561516 [ 7099 ] {08c9f7f8-c9e1-4220-a60a-63a44ca0b25a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:11:52.928174 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:52.928465 [ 7099 ] {64943803-f29e-4d30-b1d9-8073ebd55573} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46190) (comment: 00034_aggregate_key_fixed_string.sql) (in query: SELECT toFixedString(substring(SearchPhrase, 1, 17), 17) AS k1, count() AS c FROM test.hits GROUP BY k1 ORDER BY c DESC, k1 LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:52.962015 [ 7099 ] {64943803-f29e-4d30-b1d9-8073ebd55573} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:53.237910 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_41_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:53.367416 [ 4498 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:53.368125 [ 3326 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:53.368166 [ 3285 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:53.368275 [ 3312 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:53.368357 [ 3318 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:53.368462 [ 3309 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:53.368567 [ 3308 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:11:53.370355 [ 3335 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:53.370481 [ 3322 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:53.370716 [ 3297 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:53.371136 [ 3337 ] {} <Error> test_13.table_rename_with_ttl (a1b03880-5d9d-456b-9945-58f50d466a70): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:54.643238 [ 7238 ] {4620721e-1706-42a8-8e76-517cd4c49355} <Error> executeQuery: Code: 60. DB::Exception: Table default.id does not exist: While processing 1 IN (id). (UNKNOWN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46220) (comment: 02111_global_context_temporary_tables.sql) (in query: -- { echo }
 SELECT * FROM remote('127.1', system.one, 1 IN id);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
19. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
20. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
25. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
29. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
31. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0

2024.11.04 23:11:54.644778 [ 7238 ] {4620721e-1706-42a8-8e76-517cd4c49355} <Error> TCPHandler: Code: 60. DB::Exception: Table default.id does not exist: While processing 1 IN (id). (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
19. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
20. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
25. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
29. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
31. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0

2024.11.04 23:11:54.645826 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_41_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:54.662711 [ 7238 ] {1d71f983-db71-4100-abf2-50bc6fd0c405} <Error> executeQuery: Code: 60. DB::Exception: Table default.dummy does not exist: While processing 1 IN (dummy). (UNKNOWN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46220) (comment: 02111_global_context_temporary_tables.sql) (in query: SELECT * FROM remote('127.1', system.one, 1 IN dummy);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
19. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
20. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
25. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
29. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
31. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0

2024.11.04 23:11:54.663706 [ 7238 ] {1d71f983-db71-4100-abf2-50bc6fd0c405} <Error> TCPHandler: Code: 60. DB::Exception: Table default.dummy does not exist: While processing 1 IN (dummy). (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
19. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
20. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
21. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
25. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
26. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
29. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
31. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0

2024.11.04 23:11:55.598095 [ 7238 ] {0479e133-07fd-4bcd-b833-ecf62dd708b3} <Error> executeQuery: Code: 60. DB::Exception: Table default.id does not exist: While processing 1 IN (id). (UNKNOWN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46220) (comment: 02111_global_context_temporary_tables.sql) (in query: SELECT * FROM remote('127.1', view(SELECT * FROM system.one), 1 IN id);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, std::shared_ptr<DB::IAST>, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>) @ 0x0000000037c033c0
18. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, std::shared_ptr<DB::IAST> const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, std::shared_ptr<DB::IAST> const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b2768
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9b8d
20. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
21. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
25. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
26. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
28. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
30. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84

2024.11.04 23:11:55.598196 [ 3885 ] {query_id_memory_bound_merging_267397288} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:35088) (comment: 00177_memory_bound_merging.sh) (in query: SELECT URL, EventDate, max(URL) FROM remote(test_cluster_one_shard_two_replicas, test.hits) WHERE CounterID = 1704509 AND UserID = 4322253409885123546 GROUP BY CounterID, URL, EventDate ORDER BY URL, EventDate LIMIT 5 OFFSET 10 SETTINGS optimize_aggregation_in_order = 1, enable_memory_bound_merging_of_aggregation_results = 1, allow_experimental_parallel_reading_from_replicas = 1, parallel_replicas_for_non_replicated_merge_tree = 1, max_parallel_replicas = 3), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:55.599776 [ 7238 ] {0479e133-07fd-4bcd-b833-ecf62dd708b3} <Error> TCPHandler: Code: 60. DB::Exception: Table default.id does not exist: While processing 1 IN (id). (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, std::shared_ptr<DB::IAST>, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>) @ 0x0000000037c033c0
18. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, std::shared_ptr<DB::IAST> const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, std::shared_ptr<DB::IAST> const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b2768
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9b8d
20. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
21. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
25. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
26. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
28. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
30. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84

2024.11.04 23:11:55.606929 [ 3885 ] {query_id_memory_bound_merging_267397288} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:11:55.633911 [ 7238 ] {d7b1ccd1-558f-4172-9aac-c2dfd8d4238c} <Error> executeQuery: Code: 60. DB::Exception: Table default.id does not exist: While processing 1 IN (id). (UNKNOWN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46220) (comment: 02111_global_context_temporary_tables.sql) (in query: SELECT * FROM remote('127.1', view(SELECT number AS id FROM numbers(2)), 1 IN id);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, std::shared_ptr<DB::IAST>, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>) @ 0x0000000037c033c0
18. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, std::shared_ptr<DB::IAST> const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, std::shared_ptr<DB::IAST> const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b2768
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9b8d
20. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
21. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
25. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
26. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
28. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
30. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84

2024.11.04 23:11:55.635515 [ 7238 ] {d7b1ccd1-558f-4172-9aac-c2dfd8d4238c} <Error> TCPHandler: Code: 60. DB::Exception: Table default.id does not exist: While processing 1 IN (id). (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, std::vector<String, std::allocator<String>> const&, DB::SelectQueryOptions const&) @ 0x000000003660953b
7. ./build_docker/./src/Interpreters/interpretSubquery.cpp:0: DB::interpretSubquery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, unsigned long, std::vector<String, std::allocator<String>> const&) @ 0x0000000036605965
8. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::makeSet(DB::ASTFunction const&, DB::ActionsMatcher::Data&, bool) @ 0x0000000032d17b06
9. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:947: DB::ActionsMatcher::visit(DB::ASTFunction const&, std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032cf0723
10. ./build_docker/./src/Interpreters/ActionsVisitor.cpp:0: DB::ActionsMatcher::visit(std::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x0000000032ced1f5
11. ./src/Interpreters/InDepthNodeVisitor.h:78: DB::InDepthNodeVisitor<DB::ActionsMatcher, true, false, std::shared_ptr<DB::IAST> const>::doVisit(std::shared_ptr<DB::IAST> const&) @ 0x0000000032ccba15
12. ./src/Interpreters/InDepthNodeVisitor.h:0: DB::ExpressionAnalyzer::getRootActions(std::shared_ptr<DB::IAST> const&, bool, std::shared_ptr<DB::ActionsDAG>&, bool) @ 0x0000000032c451f5
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ExpressionAnalyzer::getActionsDAG(bool, bool) @ 0x0000000032c79b6b
14. ./build_docker/./src/Interpreters/ExpressionAnalyzer.cpp:1787: DB::ExpressionAnalyzer::getActions(bool, bool, DB::CompileExpressions) @ 0x0000000032c7aa93
15. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c004de
16. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
17. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, std::shared_ptr<DB::IAST>, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>) @ 0x0000000037c033c0
18. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, std::shared_ptr<DB::IAST> const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, std::shared_ptr<DB::IAST> const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b2768
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9b8d
20. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
21. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
22. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
23. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
24. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
25. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
26. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
27. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
28. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
30. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84

2024.11.04 23:11:57.771090 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_42_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:58.350724 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/twc/psfuencjepahcxhfxorqgtnqvpiac, total size 318767104, count 318767104, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id adaac87c-1c7e-49dd-abb7-0cffdc51ab98, upload has finished false.
2024.11.04 23:11:58.350951 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/twc/psfuencjepahcxhfxorqgtnqvpiac, total size 318767104, count 318767104, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id adaac87c-1c7e-49dd-abb7-0cffdc51ab98, upload has finished false
2024.11.04 23:11:58.401323 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/twc/psfuencjepahcxhfxorqgtnqvpiac, total size 318767104, count 318767104, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id adaac87c-1c7e-49dd-abb7-0cffdc51ab98, upload has finished false
2024.11.04 23:11:58.437194 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 71 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:11:59.070964 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_42_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:02.043351 [ 3757 ] {} <Error> void DB::SystemLog<DB::SessionLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::SessionLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::flushImpl(std::vector<DB::SessionLogElement, std::allocator<DB::SessionLogElement>> const&, unsigned long) @ 0x000000003604be4a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::savingThreadFunction() @ 0x00000000360485a8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f4369c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43689f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:02.075394 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_43_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:02.089191 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 92 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:03.130873 [ 3885 ] {6be736a9-38c6-4342-bc52-ff388650e0e1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:35100) (comment: 00177_memory_bound_merging.sh) (in query: SELECT COUNT() > 0 FROM system.text_log WHERE query_id IN (SELECT query_id FROM system.query_log WHERE query_id != 'query_id_memory_bound_merging_267397288' AND initial_query_id = 'query_id_memory_bound_merging_267397288' AND event_date >= yesterday()) AND event_date >= yesterday() AND message ILIKE '%Reading%ranges in order%'), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
16. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
22. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:12:03.144111 [ 3885 ] {6be736a9-38c6-4342-bc52-ff388650e0e1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
16. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
22. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:12:03.953863 [ 7238 ] {dafa683c-1d7e-447c-93a8-f331e83ed09e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=45, jobs=45). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44330) (comment: 00636_partition_key_parts_pruning.sh) (in query: SELECT count() FROM single_col_partition_key WHERE x = 20 FORMAT XML), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:12:03.959615 [ 7238 ] {dafa683c-1d7e-447c-93a8-f331e83ed09e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=45, jobs=45). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:12:03.973953 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:04.783283 [ 3948 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:46838). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/VarInt.h:92: void DB::varint_impl::readVarUInt<true>(unsigned long&, DB::ReadBuffer&) @ 0x000000001f2475c8
6. ./src/IO/ReadHelpers.h:144: DB::readStringBinary(String&, DB::ReadBuffer&, unsigned long) @ 0x000000001f2ae5d2
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3fe8
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:05.110191 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_93_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=3): (while reading column event_time_microseconds): (while reading from part /var/lib/clickhouse/disks/s3/store/590/5902c0d6-2094-4687-afb1-23195ef9ef4b/202411_1_76_14/ in table system.text_log (5902c0d6-2094-4687-afb1-23195ef9ef4b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:05.253985 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_43_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:06.208843 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 84 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:07.473110 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_43_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:08.981629 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_44_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:10.629735 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:10.629852 [ 3948 ] {28965ef0-9267-458f-95cd-1f17c9546910} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=33, jobs=33). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44426) (comment: 00636_partition_key_parts_pruning.sh) (in query: SELECT count() FROM composite_partition_key WHERE a > 400 FORMAT XML SETTINGS optimize_trivial_count_query = 0), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:10.634487 [ 3948 ] {28965ef0-9267-458f-95cd-1f17c9546910} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=33, jobs=33). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:11.522030 [ 3949 ] {e15d518f-9385-41a0-81ff-d0a03cdcc71d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44438) (comment: 00636_partition_key_parts_pruning.sh) (in query: SELECT count() FROM composite_partition_key WHERE b = 11 FORMAT XML SETTINGS optimize_trivial_count_query = 0), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:12:11.528759 [ 3949 ] {e15d518f-9385-41a0-81ff-d0a03cdcc71d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:12:12.233883 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/wvp/gllkkhydnxpbcenymjefimzhgunsu, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2051a360-a40b-4813-95b5-d2617d61a43a, upload has finished false.
2024.11.04 23:12:12.234061 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/wvp/gllkkhydnxpbcenymjefimzhgunsu, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2051a360-a40b-4813-95b5-d2617d61a43a, upload has finished false
2024.11.04 23:12:12.271678 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/wvp/gllkkhydnxpbcenymjefimzhgunsu, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2051a360-a40b-4813-95b5-d2617d61a43a, upload has finished false
2024.11.04 23:12:12.276571 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 62 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:14.735155 [ 3949 ] {392ec11a-1eda-44de-a180-6596d00550b1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44458) (comment: 00004_top_counters.sql) (in query: SELECT CounterID, count() AS c FROM test.hits GROUP BY CounterID ORDER BY c DESC LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:14.735155 [ 3948 ] {f91215e3-3643-419e-a2b8-d51f01be2a69} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45142) (comment: 00636_partition_key_parts_pruning.sh) (in query: SELECT count() FROM composite_partition_key WHERE a = 301 AND b < 20 FORMAT XML SETTINGS optimize_trivial_count_query = 0), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:14.735310 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_44_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:14.739548 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {a7e77ef5-c915-489e-89ce-773df4eb75ed::202411_1_46_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:14.739968 [ 3948 ] {f91215e3-3643-419e-a2b8-d51f01be2a69} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:14.754869 [ 3949 ] {392ec11a-1eda-44de-a180-6596d00550b1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:15.455243 [ 3949 ] {dad5189f-a20a-4737-ba02-2b4d787a74f6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45156) (comment: 00023_totals_limit.sql) (in query: SELECT goals_alias.ID AS `ym:s:goalDimension`, uniqIf(UserID, (UserID != 0) AND (`_uniq_Goals` = 1)) FROM test.visits ARRAY JOIN Goals AS goals_alias, arrayEnumerateUniq(Goals.ID) AS `_uniq_Goals` WHERE (CounterID = 842440) GROUP BY `ym:s:goalDimension` WITH TOTALS ORDER BY `ym:s:goalDimension` LIMIT 0, 1 FORMAT JSONCompact;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:15.459559 [ 3949 ] {dad5189f-a20a-4737-ba02-2b4d787a74f6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:15.594026 [ 3948 ] {57454d41-6932-44cb-a073-52718efe7622} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45158) (comment: 00636_partition_key_parts_pruning.sh) (in query: SELECT count() FROM composite_partition_key WHERE b >= 12 AND c = 2 FORMAT XML SETTINGS optimize_trivial_count_query = 0), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:15.600125 [ 3948 ] {57454d41-6932-44cb-a073-52718efe7622} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:15.899272 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:15.926896 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_44_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:16.284816 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/aiy/ffyzwkxshmbjloycxblqgtzhbugmd, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5edbe716-a9ea-40e7-a113-6ad1eb0fbccc, upload has finished false.
2024.11.04 23:12:16.284944 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/aiy/ffyzwkxshmbjloycxblqgtzhbugmd, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5edbe716-a9ea-40e7-a113-6ad1eb0fbccc, upload has finished false
2024.11.04 23:12:16.301381 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/aiy/ffyzwkxshmbjloycxblqgtzhbugmd, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5edbe716-a9ea-40e7-a113-6ad1eb0fbccc, upload has finished false
2024.11.04 23:12:16.308137 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 29 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:16.559990 [ 3948 ] {c11597b2-4b50-4b5f-8c1f-57c6a424638f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45166) (comment: 00636_partition_key_parts_pruning.sh) (in query: SELECT count() FROM composite_partition_key WHERE a = 301 AND b = 21 AND c = 3 FORMAT XML SETTINGS optimize_trivial_count_query = 0), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:16.565484 [ 3948 ] {c11597b2-4b50-4b5f-8c1f-57c6a424638f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:18.627655 [ 3949 ] {a2146f81-ec49-406b-bcc4-07e9f5f15318} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45160) (comment: 00145_aggregate_functions_statistics.sql) (in query: SELECT round(stddevSamp(ResolutionWidth), 6) FROM test.hits;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:18.629880 [ 3949 ] {a2146f81-ec49-406b-bcc4-07e9f5f15318} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:19.030067 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 67 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:19.697066 [ 3948 ] {25c1fd01-e2cd-4d85-9992-b0e401e14d1e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=29, jobs=29). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45190) (comment: 00150_quantiles_timing_precision.sql) (in query: SELECT CounterID, quantileTiming(0.5)(SendTiming) AS qt, least(30000, quantileExact(0.5)(SendTiming)) AS qe, count() AS c, round(abs(qt - qe) / greatest(qt, qe) AS diff, 3) AS rounded_diff FROM test.hits WHERE SendTiming != -1 GROUP BY CounterID HAVING diff != 0 ORDER BY diff DESC;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:19.755740 [ 3948 ] {25c1fd01-e2cd-4d85-9992-b0e401e14d1e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=29, jobs=29). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:21.236852 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 2 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:22.076362 [ 8080 ] {c2879597-ebe4-4cf8-bc0f-c1a95985e08f} <Error> executeQuery: Code: 253. DB::Exception: Replica /clickhouse/tables/db_01530_atomic/db_01530_atomic/data/replicas/test already exists. (REPLICA_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45206) (comment: 01530_drop_database_atomic_sync.sql) (in query: CREATE TABLE db_01530_atomic.data (`key` Int) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{database}/db_01530_atomic/data', 'test') ORDER BY key SETTINGS index_granularity = 38129, min_bytes_for_wide_part = 0, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = true, max_file_name_length = 0, min_bytes_for_full_part_storage = 308149961, compact_parts_max_bytes_to_buffer = 253477526, compact_parts_max_granules_to_buffer = 1, compact_parts_merge_max_bytes_to_prefetch_part = 17412280, merge_max_block_size = 15561, old_parts_lifetime = 251., prefer_fetch_merged_part_size_threshold = 5712103018, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 30, min_merge_bytes_to_use_direct_io = 10737418240, index_granularity_bytes = 16398192, concurrent_part_removal_threshold = 100, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = true, marks_compress_block_size = 21129, primary_key_compress_block_size = 64333), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::createReplica(std::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0x000000003808ab65
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:546: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions, bool) @ 0x000000003805215d
5. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageReplicatedMergeTree> std::allocate_shared[abi:v15000]<DB::StorageReplicatedMergeTree, std::allocator<DB::StorageReplicatedMergeTree>, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions&, bool&, void>(std::allocator<DB::StorageReplicatedMergeTree> const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, DB::RenamingRestrictions&, bool&) @ 0x0000000039d01af6
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2e7f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
8. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:12:22.076546 [ 3948 ] {31e04b27-083a-4e8c-8de7-d4a6d0dfcf5d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45214) (comment: 00040_aggregating_materialized_view.sql) (in query: CREATE MATERIALIZED VIEW test.basic_00040 ENGINE = AggregatingMergeTree(StartDate, (CounterID, StartDate), 8192) POPULATE AS SELECT CounterID, StartDate, sumState(Sign) AS Visits, uniqState(UserID) AS Users FROM test.visits GROUP BY CounterID, StartDate;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:12:22.077858 [ 8080 ] {c2879597-ebe4-4cf8-bc0f-c1a95985e08f} <Error> TCPHandler: Code: 253. DB::Exception: Replica /clickhouse/tables/db_01530_atomic/db_01530_atomic/data/replicas/test already exists. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::createReplica(std::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0x000000003808ab65
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:546: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions, bool) @ 0x000000003805215d
5. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageReplicatedMergeTree> std::allocate_shared[abi:v15000]<DB::StorageReplicatedMergeTree, std::allocator<DB::StorageReplicatedMergeTree>, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions&, bool&, void>(std::allocator<DB::StorageReplicatedMergeTree> const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, DB::RenamingRestrictions&, bool&) @ 0x0000000039d01af6
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2e7f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
8. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:12:22.083465 [ 3948 ] {31e04b27-083a-4e8c-8de7-d4a6d0dfcf5d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:12:22.086056 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:23.638691 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f35a2ef5-e44d-4e42-94b5-c841517c6ce9::202411_1_46_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:23.954222 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_44_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:24.268575 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_46_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:24.446433 [ 3948 ] {6ac1aa96-9c7a-4e82-ab00-c7e154e33e17} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45246) (comment: 01640_marks_corruption_regression.sql) (in query: -- This works correctly, since it does not read any marks
 SELECT 'optimize_trivial_count_query', count() FROM adaptive_table SETTINGS optimize_trivial_count_query=1 FORMAT CSV;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:24.448109 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:24.448640 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f35a2ef5-e44d-4e42-94b5-c841517c6ce9::202411_1_46_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:24.451505 [ 3948 ] {6ac1aa96-9c7a-4e82-ab00-c7e154e33e17} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:25.461692 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_46_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:25.579116 [ 3948 ] {46bd0f5e-67b0-4dfe-bb71-cfee4b89500f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40344) (comment: 02922_respect_nulls_states.sql) (in query: SELECT first_value_respect_nullsMerge(t) FROM (Select first_value_respect_nullsState(NULL::Nullable(UInt8)) as t FROM numbers(5));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:25.584554 [ 3948 ] {46bd0f5e-67b0-4dfe-bb71-cfee4b89500f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:26.909007 [ 8081 ] {5d48f086-1f9c-4472-ab00-6789bd763b53} <Error> executeQuery: Code: 387. DB::Exception: Dictionary test_13.test_dict already exists. (DICTIONARY_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40396) (comment: 02973_dictionary_table_exception_fix.sql) (in query: CREATE TABLE test_dict (`y` Int64) ENGINE = MergeTree ORDER BY y SETTINGS index_granularity = 63164, min_bytes_for_wide_part = 0, ratio_of_defaults_for_sparse_serialization = 0.9010027647018433, replace_long_file_name_to_hash = true, max_file_name_length = 128, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 412204793, compact_parts_max_granules_to_buffer = 1, compact_parts_merge_max_bytes_to_prefetch_part = 32906516, merge_max_block_size = 8173, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 5162965048, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 10250363191, index_granularity_bytes = 18422147, concurrent_part_removal_threshold = 0, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 19340, primary_key_compress_block_size = 75304), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1392: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466720d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:26.910387 [ 8081 ] {5d48f086-1f9c-4472-ab00-6789bd763b53} <Error> TCPHandler: Code: 387. DB::Exception: Dictionary test_13.test_dict already exists. (DICTIONARY_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1392: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466720d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:26.915312 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_47_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:27.469211 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ybx/izmlkqchagxuffvgxaxprhjqbgmnc, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id de2e5c54-35df-4795-84d0-27f33a7f43c6, upload has finished false.
2024.11.04 23:12:27.469366 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ybx/izmlkqchagxuffvgxaxprhjqbgmnc, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id de2e5c54-35df-4795-84d0-27f33a7f43c6, upload has finished false
2024.11.04 23:12:27.479892 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ybx/izmlkqchagxuffvgxaxprhjqbgmnc, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id de2e5c54-35df-4795-84d0-27f33a7f43c6, upload has finished false
2024.11.04 23:12:27.673157 [ 8081 ] {dc19da73-5baf-4f24-a9b3-576720825bc8} <Error> executeQuery: Code: 57. DB::Exception: Table test_13.test_table already exists. (TABLE_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40396) (comment: 02973_dictionary_table_exception_fix.sql) (in query: CREATE DICTIONARY test_table (y String, value UInt64 DEFAULT 0) PRIMARY KEY y SOURCE(CLICKHOUSE(TABLE 'test_table')) LAYOUT(DIRECT());), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:27.673290 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 57 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:27.674226 [ 8081 ] {dc19da73-5baf-4f24-a9b3-576720825bc8} <Error> TCPHandler: Code: 57. DB::Exception: Table test_13.test_table already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:27.684323 [ 8081 ] {b722bead-0407-48d1-9c3b-9b1d78391b30} <Error> executeQuery: Code: 387. DB::Exception: Dictionary test_13.test_dict already exists. (DICTIONARY_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40396) (comment: 02973_dictionary_table_exception_fix.sql) (in query: CREATE DICTIONARY test_dict (y String, value UInt64 DEFAULT 0) PRIMARY KEY y SOURCE(CLICKHOUSE(TABLE 'test_table')) LAYOUT(DIRECT());), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1392: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466720d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:27.685297 [ 8081 ] {b722bead-0407-48d1-9c3b-9b1d78391b30} <Error> TCPHandler: Code: 387. DB::Exception: Dictionary test_13.test_dict already exists. (DICTIONARY_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1392: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466720d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:27.697664 [ 8081 ] {beaeba74-2183-48d0-8ea6-277d869db54f} <Error> executeQuery: Code: 57. DB::Exception: Table test_13.test_table already exists. (TABLE_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40396) (comment: 02973_dictionary_table_exception_fix.sql) (in query: CREATE TABLE test_table (`y` Int64) ENGINE = MergeTree ORDER BY y SETTINGS index_granularity = 63164, min_bytes_for_wide_part = 0, ratio_of_defaults_for_sparse_serialization = 0.9010027647018433, replace_long_file_name_to_hash = true, max_file_name_length = 128, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 412204793, compact_parts_max_granules_to_buffer = 1, compact_parts_merge_max_bytes_to_prefetch_part = 32906516, merge_max_block_size = 8173, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 5162965048, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 10250363191, index_granularity_bytes = 18422147, concurrent_part_removal_threshold = 0, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 19340, primary_key_compress_block_size = 75304), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:27.698546 [ 8081 ] {beaeba74-2183-48d0-8ea6-277d869db54f} <Error> TCPHandler: Code: 57. DB::Exception: Table test_13.test_table already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:12:27.717880 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_47_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:28.831225 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:28.843084 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_99_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=29). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:686: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000394021b3
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:28.847415 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_47_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:29.892836 [ 8081 ] {81e51276-22ad-44bb-8535-13d255f73427} <Error> executeQuery: Code: 42. DB::Exception: Table name was not found in function arguments. Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: select * from remote('127.0.0.1', sys);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const>(int, FormatStringHelperImpl<std::type_identity<String const>::type>, String const&&) @ 0x000000002f4ae7b5
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:178: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6afc
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:29.894162 [ 8081 ] {81e51276-22ad-44bb-8535-13d255f73427} <Error> TCPHandler: Code: 42. DB::Exception: Table name was not found in function arguments. Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const>(int, FormatStringHelperImpl<std::type_identity<String const>::type>, String const&&) @ 0x000000002f4ae7b5
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:178: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6afc
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:29.906670 [ 8081 ] {f0e4758e-8f80-4079-977e-69757138f6db} <Error> executeQuery: Code: 42. DB::Exception: Table name was not found in function arguments. Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: select * from remote('127.0.0.1', system);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const>(int, FormatStringHelperImpl<std::type_identity<String const>::type>, String const&&) @ 0x000000002f4ae7b5
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:178: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6afc
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:29.907546 [ 8081 ] {f0e4758e-8f80-4079-977e-69757138f6db} <Error> TCPHandler: Code: 42. DB::Exception: Table name was not found in function arguments. Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const>(int, FormatStringHelperImpl<std::type_identity<String const>::type>, String const&&) @ 0x000000002f4ae7b5
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:178: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6afc
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:30.742713 [ 8081 ] {421b5b78-35fe-4a0e-9834-23817bdfce84} <Error> executeQuery: Code: 60. DB::Exception: Table system.o does not exist. Maybe you meant test_13.t?. (UNKNOWN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: select * from remote('127.0.0.1', system.o);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&, String&&) @ 0x0000000007ecad7f
3. ./build_docker/./src/Databases/IDatabase.cpp:39: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f8252b
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:53: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x00000000387355f7
7. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
9. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a91a9
10. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
11. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:12:30.745078 [ 8081 ] {421b5b78-35fe-4a0e-9834-23817bdfce84} <Error> TCPHandler: Code: 60. DB::Exception: Table system.o does not exist. Maybe you meant test_13.t?. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&, String&&) @ 0x0000000007ecad7f
3. ./build_docker/./src/Databases/IDatabase.cpp:39: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f8252b
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:53: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x00000000387355f7
7. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
9. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a91a9
10. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
11. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:12:30.753926 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_47_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:31.771109 [ 8081 ] {a8b0c4c1-26de-4da7-9adb-4ee875a711af} <Error> executeQuery: Code: 47. DB::Exception: Missing columns: 'default' while processing query: 'default', required columns: 'default' 'default'. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: select * from remote('127.0.0.1', system.one, default);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/StorageDistributed.cpp:222: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c00101
6. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
7. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
9. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
10. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:12:31.771359 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 13 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:31.772622 [ 8081 ] {a8b0c4c1-26de-4da7-9adb-4ee875a711af} <Error> TCPHandler: Code: 47. DB::Exception: Missing columns: 'default' while processing query: 'default', required columns: 'default' 'default'. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/StorageDistributed.cpp:222: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c00101
6. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
7. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
9. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
10. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:12:31.775042 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_47_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:32.773817 [ 8081 ] {7699a764-ed59-4f0a-9770-c681a35e99c0} <Error> executeQuery: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: SELECT * FROM remote('127.0.0.1', system.one, default, '[HIDDEN]')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.775285 [ 8081 ] {7699a764-ed59-4f0a-9770-c681a35e99c0} <Error> TCPHandler: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.786356 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=1): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:32.789204 [ 8081 ] {389ffc92-b04b-447f-9256-d6e89121c181} <Error> executeQuery: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: select * from remote('127.0.0.1', system.one, default, key1);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.790194 [ 8081 ] {389ffc92-b04b-447f-9256-d6e89121c181} <Error> TCPHandler: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.804209 [ 8081 ] {38c8c9ec-97b7-4849-8f39-3453d6ac6155} <Error> executeQuery: Code: 47. DB::Exception: Missing columns: 'key1' while processing query: 'key1', required columns: 'key1' 'key1'. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: SELECT * FROM remote('127.0.0.1', system.one, 'default', '[HIDDEN]', key1)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/StorageDistributed.cpp:222: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c00101
6. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
7. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
9. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
10. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.805465 [ 8081 ] {38c8c9ec-97b7-4849-8f39-3453d6ac6155} <Error> TCPHandler: Code: 47. DB::Exception: Missing columns: 'key1' while processing query: 'key1', required columns: 'key1' 'key1'. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/StorageDistributed.cpp:222: DB::(anonymous namespace)::buildShardingKeyExpression(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, bool) @ 0x0000000037c00101
6. ./build_docker/./src/Storages/StorageDistributed.cpp:0: DB::StorageDistributed::StorageDistributed(DB::StorageID const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&, String const&, String const&, DB::DistributedSettings const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster>, std::shared_ptr<DB::IAST>) @ 0x0000000037bf9167
7. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageDistributed> std::allocate_shared[abi:v15000]<DB::StorageDistributed, std::allocator<DB::StorageDistributed>, DB::StorageID, DB::ColumnsDescription&, DB::ConstraintsDescription, String, String const&, String const&, String, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String, String, DB::DistributedSettings, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Cluster> const&, void>(std::allocator<DB::StorageDistributed> const&, DB::StorageID&&, DB::ColumnsDescription&, DB::ConstraintsDescription&&, String&&, String const&, String const&, String&&, std::shared_ptr<DB::Context const>&, std::shared_ptr<DB::IAST> const&, String&&, String&&, DB::DistributedSettings&&, DB::LoadingStrictnessLevel&&, std::shared_ptr<DB::Cluster> const&) @ 0x000000002f4b33dc
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a9ffd
9. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
10. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.853529 [ 8081 ] {192484a2-a07c-4b9e-aa87-6ae911c074ea} <Error> executeQuery: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: SELECT * FROM remote('127.0.0.1', system.one, default, '[HIDDEN]', key1)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.854681 [ 8081 ] {192484a2-a07c-4b9e-aa87-6ae911c074ea} <Error> TCPHandler: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.938625 [ 8081 ] {4e292636-037b-4765-bb1f-aeacb07fe3bd} <Error> executeQuery: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: select * from remote('127.0.0.1', system.one, 'default', pwd, key1);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:32.939713 [ 8081 ] {4e292636-037b-4765-bb1f-aeacb07fe3bd} <Error> TCPHandler: Code: 36. DB::Exception: Arguments `user` and `password` should be string literals (in single quotes). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a6954
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:33.779903 [ 8081 ] {4ea209b5-776c-4aa9-8367-764a10c47d4c} <Error> executeQuery: Code: 42. DB::Exception: Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: SELECT * FROM remote('127.0.0.1', system.one, 'default', '[HIDDEN]', key1, key2)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage const&, int) @ 0x000000002f4ae37b
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a02c4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:35.910242 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:35.910242 [ 8081 ] {4ea209b5-776c-4aa9-8367-764a10c47d4c} <Error> TCPHandler: Code: 42. DB::Exception: Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage const&, int) @ 0x000000002f4ae37b
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a02c4
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:36.610997 [ 8081 ] {bf71ec14-6357-4eab-94c4-28737e5beffb} <Error> executeQuery: Code: 42. DB::Exception: Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40440) (comment: 02841_remote_parameter_parsing_error.sql) (in query: SELECT * FROM remote('127.0.0.1', system, one, 'default', '[HIDDEN]', key1, key2)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage const&, int) @ 0x000000002f4ae37b
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:118: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a64ff
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:36.612282 [ 8081 ] {bf71ec14-6357-4eab-94c4-28737e5beffb} <Error> TCPHandler: Code: 42. DB::Exception: Table function 'remote' requires from 1 to 6 parameters: <addresses pattern> [, <name of remote database>, <name of remote table>] [, username[, password], sharding_key]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage const&, int) @ 0x000000002f4ae37b
3. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:118: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a64ff
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347c96b8
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:12:36.628671 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_47_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:37.204655 [ 3948 ] {9e28202d-27be-45f8-8497-5e77ef161103} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=34, jobs=34). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45472) (comment: 02710_topk_with_empty_array.sql) (in query: SELECT topK(emptyArrayInt16());), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:37.208450 [ 3948 ] {9e28202d-27be-45f8-8497-5e77ef161103} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=34, jobs=34). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:37.261133 [ 8080 ] {c48382e8-dd0d-4ef9-b82b-117d1e20dfb6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45478) (comment: 00083_array_filter.sql) (in query: SELECT sum(length(ParsedParams.Key1)) FROM test.hits WHERE notEmpty(ParsedParams.Key1);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:37.263016 [ 8080 ] {c48382e8-dd0d-4ef9-b82b-117d1e20dfb6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:38.803571 [ 3948 ] {80fae34c-9714-4596-972a-82ac5eb58f6a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45482) (comment: 01710_projection_part_check.sql) (in query: insert into tp select number, number from numbers(3);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeProjectionPartImpl(String const&, bool, DB::IMergeTreeDataPart*, DB::MergeTreeData const&, std::shared_ptr<Poco::Logger>, DB::Block, DB::ProjectionDescription const&) @ 0x0000000039973af7
15. ./src/Core/Block.h:35: DB::MergeTreeDataWriter::writeProjectionPart(DB::MergeTreeData const&, std::shared_ptr<Poco::Logger>, DB::Block, DB::ProjectionDescription const&, DB::IMergeTreeDataPart*) @ 0x000000003996f88f
16. ./src/Core/Block.h:35: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039965dfa
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
25. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
26. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
27. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:12:38.803638 [ 8080 ] {3c85e6fa-5388-45fb-831f-54a4fcdd7814} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45494) (comment: 00148_monotonic_functions_and_index.sql) (in query: SELECT count() FROM test.hits WHERE abs(-CounterID) = 1731;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:39.861975 [ 3948 ] {80fae34c-9714-4596-972a-82ac5eb58f6a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeProjectionPartImpl(String const&, bool, DB::IMergeTreeDataPart*, DB::MergeTreeData const&, std::shared_ptr<Poco::Logger>, DB::Block, DB::ProjectionDescription const&) @ 0x0000000039973af7
15. ./src/Core/Block.h:35: DB::MergeTreeDataWriter::writeProjectionPart(DB::MergeTreeData const&, std::shared_ptr<Poco::Logger>, DB::Block, DB::ProjectionDescription const&, DB::IMergeTreeDataPart*) @ 0x000000003996f88f
16. ./src/Core/Block.h:35: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039965dfa
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
25. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
26. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
27. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:12:39.862057 [ 8080 ] {3c85e6fa-5388-45fb-831f-54a4fcdd7814} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:39.862293 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_48_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:39.897910 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ebw/pjyhgvvjbglgkiarjutajktuobtbd, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 538ac296-c811-4eaa-8deb-4562471f14d0, upload has finished false.
2024.11.04 23:12:39.898021 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ebw/pjyhgvvjbglgkiarjutajktuobtbd, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 538ac296-c811-4eaa-8deb-4562471f14d0, upload has finished false
2024.11.04 23:12:39.949463 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ebw/pjyhgvvjbglgkiarjutajktuobtbd, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 538ac296-c811-4eaa-8deb-4562471f14d0, upload has finished false
2024.11.04 23:12:41.293661 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:41.294109 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. ./src/IO/WriteBuffer.h:65: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039408090
17. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&) @ 0x00000000393f910f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f5c43
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:41.297321 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_48_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:41.387221 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {33f0dc8f-1542-43c0-a136-e415d11993b4::202411_1_47_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:43.314827 [ 8080 ] {2603406c-5d59-45e0-bf38-1f66fc8ce8f2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=47, jobs=47). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45508) (comment: 01881_join_on_conditions_hash.gen.sql) (in query: SELECT t1.key, t1.key2 FROM t1 INNER ALL JOIN t2 ON t1.id == t2.id AND t2.key == t2.key2 AND t1.key == t1.key2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:43.371527 [ 8080 ] {2603406c-5d59-45e0-bf38-1f66fc8ce8f2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=47, jobs=47). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:43.974631 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:45.154088 [ 8080 ] {78f60211-7068-46f1-87a6-3f2d09bf1abe} <Error> executeQuery: Code: 153. DB::Exception: Division by zero: while executing 'FUNCTION intDiv(__table1.b :: 2, __table1.c :: 0) -> intDiv(__table1.b, __table1.c) Int32 : 1'. (ILLEGAL_DIVISION) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39000) (comment: 02481_prewhere_filtered_rows_div_by_zero.sql) (in query: -- Check that division by zero occurs on some rows
 SELECT intDiv(b, c) FROM test_filter;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. void DB::throwIfDivisionLeadsToFPE<int, int>(int, int) @ 0x000000000d4087b1
4. void DB::(anonymous namespace)::DivideIntegralByConstantImpl<int, int>::process<(DB::impl_::OpCase)0>(int const*, int const*, int*, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x0000000011d459fc
5. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeNumeric<DB::DataTypeNumber<int>, DB::DataTypeNumber<int>>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, DB::DataTypeNumber<int> const&, DB::DataTypeNumber<int> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const @ 0x00000000124e4351
6. bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castType<auto bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castBothTypes<DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl2(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const::'lambda'(auto const&, auto const&)>(DB::IDataType const*, DB::IDataType const*, auto&&)::'lambda'(auto const&)::operator()<DB::DataTypeNumber<int>>(auto const&) const::'lambda'(auto const&)>(DB::IDataType const*, auto&&) @ 0x00000000124d0bcf
7. bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castType<bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castBothTypes<DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl2(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const::'lambda'(auto const&, auto const&)>(DB::IDataType const*, DB::IDataType const*, auto&&)::'lambda'(auto const&)>(DB::IDataType const*, auto&&) @ 0x000000001235831d
8. DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl2(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const @ 0x00000000123481de
9. DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000012342814
10. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede698
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c39d
12. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:124: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x0000000032bf1992
15. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:0: DB::ExpressionTransform::transform(DB::Chunk&) @ 0x000000003afffa5e
16. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
17. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
18. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
19. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
22. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:45.158961 [ 8080 ] {78f60211-7068-46f1-87a6-3f2d09bf1abe} <Error> TCPHandler: Code: 153. DB::Exception: Division by zero: while executing 'FUNCTION intDiv(__table1.b :: 2, __table1.c :: 0) -> intDiv(__table1.b, __table1.c) Int32 : 1'. (ILLEGAL_DIVISION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. void DB::throwIfDivisionLeadsToFPE<int, int>(int, int) @ 0x000000000d4087b1
4. void DB::(anonymous namespace)::DivideIntegralByConstantImpl<int, int>::process<(DB::impl_::OpCase)0>(int const*, int const*, int*, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x0000000011d459fc
5. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeNumeric<DB::DataTypeNumber<int>, DB::DataTypeNumber<int>>(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, DB::DataTypeNumber<int> const&, DB::DataTypeNumber<int> const&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const @ 0x00000000124e4351
6. bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castType<auto bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castBothTypes<DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl2(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const::'lambda'(auto const&, auto const&)>(DB::IDataType const*, DB::IDataType const*, auto&&)::'lambda'(auto const&)::operator()<DB::DataTypeNumber<int>>(auto const&) const::'lambda'(auto const&)>(DB::IDataType const*, auto&&) @ 0x00000000124d0bcf
7. bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castType<bool DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::castBothTypes<DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl2(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const::'lambda'(auto const&, auto const&)>(DB::IDataType const*, DB::IDataType const*, auto&&)::'lambda'(auto const&)>(DB::IDataType const*, auto&&) @ 0x000000001235831d
8. DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl2(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) const @ 0x00000000123481de
9. DB::FunctionBinaryArithmetic<DB::DivideIntegralImpl, DB::NameIntDiv, false, true, false>::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000012342814
10. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x0000000008ede698
11. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4c39d
12. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::executeWithoutSparseColumns(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd4e2d7
13. ./build_docker/./src/Functions/IFunction.cpp:0: DB::IExecutableFunction::execute(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, std::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0x000000002fd55226
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:124: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x0000000032bf1992
15. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:0: DB::ExpressionTransform::transform(DB::Chunk&) @ 0x000000003afffa5e
16. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000025370d21
17. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000003a62d5fe
18. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
19. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
22. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:12:45.190333 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:12:45.190420 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:12:46.076326 [ 3948 ] {cc29b452-543e-427b-a9fe-22eff48ffbdf} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45496) (comment: 00167_read_bytes_from_fs.sql) (in query: SELECT sum(cityHash64(*)) FROM test.hits SETTINGS max_threads=40;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:12:46.626085 [ 3948 ] {cc29b452-543e-427b-a9fe-22eff48ffbdf} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:12:46.742014 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_48_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:46.913420 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ppk/cpuhlkdyvyomavwihlwowvwtuzrah, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c921de31-45c3-4e0c-865e-d12d065639e4, upload has finished false.
2024.11.04 23:12:46.913571 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ppk/cpuhlkdyvyomavwihlwowvwtuzrah, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c921de31-45c3-4e0c-865e-d12d065639e4, upload has finished false
2024.11.04 23:12:46.921852 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ppk/cpuhlkdyvyomavwihlwowvwtuzrah, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c921de31-45c3-4e0c-865e-d12d065639e4, upload has finished false
2024.11.04 23:12:46.983467 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 8 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:47.043269 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:47.581974 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_49_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:47.583127 [ 3948 ] {060bfe6a-2a21-49c6-8de8-8a22fa820529} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39024) (comment: 00068_subquery_in_prewhere.sql) (in query: SELECT count() FROM test.hits PREWHERE UserID IN (SELECT UserID FROM test.hits WHERE CounterID = 800784);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:47.592044 [ 3948 ] {060bfe6a-2a21-49c6-8de8-8a22fa820529} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:47.966520 [ 3927 ] {8d00faec-620c-44b2-9333-f82d871f0e29} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=967). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53982) (comment: 00858_issue_4756.sql) (in query: SELECT DISTINCT `__table1`.`id` AS `id` FROM `test_13`.`shard1` AS `__table1` WHERE `__table1`.`id` IN (SELECT `__table1`.`id` AS `id` FROM `test_13`.`shard1` AS `__table1` ALL INNER JOIN `test_13`.`shard2` AS `__table2` ON `__table1`.`id` = `__table2`.`id` WHERE `__table1`.`id` > 0)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:12:47.969854 [ 3927 ] {8d00faec-620c-44b2-9333-f82d871f0e29} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=967). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:12:47.973989 [ 8080 ] {440df921-b1bb-418f-9806-d775a0102969} <Error> executeQuery: Code: 439. DB::Exception: Received from localhost:9000. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=967). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39014) (comment: 00858_issue_4756.sql) (in query: select distinct(distr1.id) from distr1 where distr1.id in ( select distr1.id from distr1 join distr2 on distr1.id = distr2.id where distr1.id > 0 );), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:12:47.978443 [ 8080 ] {440df921-b1bb-418f-9806-d775a0102969} <Error> TCPHandler: Code: 439. DB::Exception: Received from localhost:9000. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=967). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
: While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:12:48.205954 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 19 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:48.339237 [ 3948 ] {cdb1a35b-456d-4a58-a7ee-dafaaabc79cf} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39040) (comment: 00001_count_hits.sql) (in query: SELECT count() FROM test.hits ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:48.343040 [ 3948 ] {cdb1a35b-456d-4a58-a7ee-dafaaabc79cf} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:50.481657 [ 3948 ] {7bbaa9fc-7d4c-4584-ae22-1ef7402f636f} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.87 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39050) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 = (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6eec04
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003482f1fd
11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347bde43
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003480890d
14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86

2024.11.04 23:12:50.499331 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_9_9_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:50.500146 [ 3948 ] {7bbaa9fc-7d4c-4584-ae22-1ef7402f636f} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.87 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6eec04
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::evaluateScalarSubqueryIfNeeded(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003482f1fd
11. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347bde43
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003480890d
14. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
15. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
16. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
17. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
18. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
25. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
26. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86

2024.11.04 23:12:51.373032 [ 3948 ] {7a603f2b-dee1-426a-a6f5-8ba89a896918} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.87 million. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39050) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT 1 IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6eec04
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:12:52.426939 [ 3948 ] {7a603f2b-dee1-426a-a6f5-8ba89a896918} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 1.00 million, current rows: 8.87 million. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6eec04
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:12:52.427992 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:52.470659 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {a7e77ef5-c915-489e-89ce-773df4eb75ed::202411_1_51_10}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:52.471521 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_49_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:53.596480 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_50_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:55.854669 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:55.855115 [ 3948 ] {a149a270-e7bf-4fbe-8429-2ea292f9372d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39050) (comment: 00175_counting_resources_in_subqueries.sql) (in query: SELECT count() IN (SELECT count() FROM test.hits WHERE NOT ignore(AdvEngineID)) FROM test.hits WHERE NOT ignore(AdvEngineID);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:12:55.858377 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_50_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:55.933662 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 96 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:56.231677 [ 3948 ] {a149a270-e7bf-4fbe-8429-2ea292f9372d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:12:57.017488 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:58.948619 [ 3948 ] {66f49a74-16ca-459e-aad0-2811c8554461} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56834) (comment: 00021_3_select_with_in.sql) (in query: SELECT count() FROM remote('localhost', test, hits) WHERE CounterID IN (598875);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:12:58.948680 [ 3767 ] {} <Error> PushingToViews: Cannot push to the storage, ignoring the error: Code: 60. DB::Exception: Table test_13.receiver_02572 does not exist: while pushing to view test_13.push_to_logs_proxy_mv_02572 (e2fde351-8d88-4eae-ae26-174dfab64963). (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x00000000355aa545
7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:406: DB::InterpreterInsertQuery::execute() @ 0x00000000355b7967
8. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::writeToLocal(DB::Cluster::ShardInfo const&, DB::Block const&, unsigned long) @ 0x0000000038d5c2f6
9. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::writeAsyncImpl(DB::Block const&, unsigned long) @ 0x0000000038d4bcdc
10. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::consume(DB::Chunk) @ 0x0000000038d47b28
11. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
12. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
13. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
14. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
15. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
16. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
17. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
18. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
19. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
20. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
21. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:58.966725 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_50_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:58.993524 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 25 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:12:59.009034 [ 3948 ] {66f49a74-16ca-459e-aad0-2811c8554461} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:00.559916 [ 3948 ] {8ef3b2e4-f1f2-488f-83b6-07ee53f4647a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=0): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56860) (comment: 00043_any_left_join.sql) (in query: SELECT EventDate, count() AS hits, any(visits) FROM test.hits ANY LEFT JOIN ( SELECT StartDate AS EventDate, sum(Sign) AS visits FROM test.visits GROUP BY EventDate ) USING EventDate GROUP BY EventDate ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003b75a78e
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003b73fe83
8. ./contrib/llvm-project/libcxx/include/vector:1562: DB::MergeTreePrefetchedReadPool::PrefetchedReaders::PrefetchedReaders(DB::MergeTreeReadTask::Readers, Priority, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b73e97d
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::MergeTreePrefetchedReadPool::PrefetchedReaders>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreePrefetchedReadPool::PrefetchedReaders, DB::MergeTreeReadTask::Readers, Priority&, DB::MergeTreePrefetchedReadPool&>(DB::MergeTreeReadTask::Readers&&, Priority&, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b75d6fc
10. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:167: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b750566
11. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
12. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
13. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
14. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
15. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
16. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
17. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
19. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
20. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
21. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:13:00.649080 [ 3948 ] {8ef3b2e4-f1f2-488f-83b6-07ee53f4647a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=0): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003b75a78e
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003b73fe83
8. ./contrib/llvm-project/libcxx/include/vector:1562: DB::MergeTreePrefetchedReadPool::PrefetchedReaders::PrefetchedReaders(DB::MergeTreeReadTask::Readers, Priority, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b73e97d
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::MergeTreePrefetchedReadPool::PrefetchedReaders>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreePrefetchedReadPool::PrefetchedReaders, DB::MergeTreeReadTask::Readers, Priority&, DB::MergeTreePrefetchedReadPool&>(DB::MergeTreeReadTask::Readers&&, Priority&, DB::MergeTreePrefetchedReadPool&) @ 0x000000003b75d6fc
10. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:167: DB::MergeTreePrefetchedReadPool::createPrefetchedReadersForTask(DB::MergeTreePrefetchedReadPool::ThreadTask&) @ 0x000000003b750566
11. ./contrib/llvm-project/libcxx/include/vector:1420: DB::MergeTreePrefetchedReadPool::startPrefetches() @ 0x000000003b750c97
12. ./build_docker/./src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp:0: DB::MergeTreePrefetchedReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b7514c2
13. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
14. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
15. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
16. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
17. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
19. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
20. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
21. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:13:00.935189 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 81 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:01.345642 [ 3948 ] {db2432fd-6781-4b2b-95f7-014fe371feaa} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56874) (comment: 00036_aggregate_hashed.sql) (in query: SELECT SearchEngineID AS k1, SearchPhrase AS k2, count() AS c FROM test.hits GROUP BY k1, k2 ORDER BY c DESC, k1, k2 LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:01.374444 [ 3948 ] {db2432fd-6781-4b2b-95f7-014fe371feaa} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:02.088743 [ 3948 ] {ba96363b-0567-4c63-b9c1-6bba05b73874} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56876) (comment: 00007_uniq.sql) (in query: SELECT RegionID, uniq(UserID) AS u FROM test.hits WHERE CounterID = 800784 GROUP BY RegionID ORDER BY u DESC, RegionID LIMIT 10 -- nothing
 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:13:02.091020 [ 3948 ] {ba96363b-0567-4c63-b9c1-6bba05b73874} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:13:02.184528 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 13 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:02.184785 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_50_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:03.370428 [ 3948 ] {86517c5a-527b-49f4-b8b0-b8aefa1e4d82} <Error> executeQuery: Code: 630. DB::Exception: Cannot drop or rename test.hits, because some tables depend on it: db_dict.cache_hits. (HAVE_DEPENDENT_OBJECTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56884) (comment: 00140_rename.sql) (in query: RENAME TABLE test.hits TO test.visits_tmp, test.visits TO test.hits, test.visits_tmp TO test.visits;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<DB::StorageID const&, fmt::v8::join_view<std::__wrap_iter<DB::StorageID*>, std::__wrap_iter<DB::StorageID*>, char>>(int, FormatStringHelperImpl<std::type_identity<DB::StorageID const&>::type, std::type_identity<fmt::v8::join_view<std::__wrap_iter<DB::StorageID*>, std::__wrap_iter<DB::StorageID*>, char>>::type>, DB::StorageID const&, fmt::v8::join_view<std::__wrap_iter<DB::StorageID*>, std::__wrap_iter<DB::StorageID*>, char>&&) @ 0x0000000032bbf9a2
3. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:1456: DB::DatabaseCatalog::checkTableCanBeRemovedOrRenamedUnlocked(DB::StorageID const&, bool, bool, bool) const @ 0x0000000032ba9740
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::removeDependencies(DB::StorageID const&, bool, bool, bool) @ 0x0000000032ba8605
5. ./build_docker/./src/Interpreters/InterpreterRenameQuery.cpp:0: DB::InterpreterRenameQuery::executeToTables(DB::ASTRenameQuery const&, std::vector<DB::RenameDescription, std::allocator<DB::RenameDescription>> const&, std::map<DB::UniqueTableName, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>, std::less<DB::UniqueTableName>, std::allocator<std::pair<DB::UniqueTableName const, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>>>>&) @ 0x00000000355def0e
6. ./build_docker/./src/Interpreters/InterpreterRenameQuery.cpp:0: DB::InterpreterRenameQuery::execute() @ 0x00000000355d89fd
7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
8. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:13:03.370534 [ 3767 ] {} <Error> PushingToViews: Cannot push to the storage, ignoring the error: Code: 60. DB::Exception: Table test_13.receiver_02572 does not exist: while pushing to view test_13.push_to_logs_proxy_mv_02572 (e2fde351-8d88-4eae-ae26-174dfab64963). (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000031f821cb
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:429: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b774a5
5. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
6. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x00000000355aa545
7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:406: DB::InterpreterInsertQuery::execute() @ 0x00000000355b7967
8. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::writeToLocal(DB::Cluster::ShardInfo const&, DB::Block const&, unsigned long) @ 0x0000000038d5c2f6
9. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::writeAsyncImpl(DB::Block const&, unsigned long) @ 0x0000000038d4bcdc
10. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::consume(DB::Chunk) @ 0x0000000038d47b28
11. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
12. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
13. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
14. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
15. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
16. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
17. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
18. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
19. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
20. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
21. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
23. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:03.373798 [ 3948 ] {86517c5a-527b-49f4-b8b0-b8aefa1e4d82} <Error> TCPHandler: Code: 630. DB::Exception: Cannot drop or rename test.hits, because some tables depend on it: db_dict.cache_hits. (HAVE_DEPENDENT_OBJECTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<DB::StorageID const&, fmt::v8::join_view<std::__wrap_iter<DB::StorageID*>, std::__wrap_iter<DB::StorageID*>, char>>(int, FormatStringHelperImpl<std::type_identity<DB::StorageID const&>::type, std::type_identity<fmt::v8::join_view<std::__wrap_iter<DB::StorageID*>, std::__wrap_iter<DB::StorageID*>, char>>::type>, DB::StorageID const&, fmt::v8::join_view<std::__wrap_iter<DB::StorageID*>, std::__wrap_iter<DB::StorageID*>, char>&&) @ 0x0000000032bbf9a2
3. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:1456: DB::DatabaseCatalog::checkTableCanBeRemovedOrRenamedUnlocked(DB::StorageID const&, bool, bool, bool) const @ 0x0000000032ba9740
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::removeDependencies(DB::StorageID const&, bool, bool, bool) @ 0x0000000032ba8605
5. ./build_docker/./src/Interpreters/InterpreterRenameQuery.cpp:0: DB::InterpreterRenameQuery::executeToTables(DB::ASTRenameQuery const&, std::vector<DB::RenameDescription, std::allocator<DB::RenameDescription>> const&, std::map<DB::UniqueTableName, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>, std::less<DB::UniqueTableName>, std::allocator<std::pair<DB::UniqueTableName const, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>>>>&) @ 0x00000000355def0e
6. ./build_docker/./src/Interpreters/InterpreterRenameQuery.cpp:0: DB::InterpreterRenameQuery::execute() @ 0x00000000355d89fd
7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
8. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:13:04.328930 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_61_12}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:04.805235 [ 3948 ] {7870442e-cb0e-409f-8aa6-deecc7b1a3f3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57238) (comment: 00170_s3_cache.sql) (in query: SELECT sum(AdvEngineID), count(), avg(ResolutionWidth) FROM test.hits_s3 ;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:04.810968 [ 3948 ] {7870442e-cb0e-409f-8aa6-deecc7b1a3f3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:05.779742 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_52_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:05.955934 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:13:05.956043 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:13:06.967101 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_52_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:07.647928 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/zxb/zhhskisisxgnvbndvcpusuhvgrmtr, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 152398f7-c1ed-425e-9e65-63199d8c1cbd, upload has finished false.
2024.11.04 23:13:07.648092 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/zxb/zhhskisisxgnvbndvcpusuhvgrmtr, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 152398f7-c1ed-425e-9e65-63199d8c1cbd, upload has finished false
2024.11.04 23:13:07.657358 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/zxb/zhhskisisxgnvbndvcpusuhvgrmtr, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 152398f7-c1ed-425e-9e65-63199d8c1cbd, upload has finished false
2024.11.04 23:13:07.661019 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 49 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:08.327863 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_54_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=27). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:09.236663 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:09.242763 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_100_110_2}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:09.674552 [ 3948 ] {9b1a01e0-7b36-4f01-8034-83587f67ee0b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.051474964 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57252) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:09.799054 [ 3948 ] {9b1a01e0-7b36-4f01-8034-83587f67ee0b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.051474964 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:10.840983 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_54_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:11.399225 [ 3948 ] {67a6c7f4-505d-421f-9823-8294a5a983c7} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.061269183 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57256) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:11.480796 [ 3948 ] {67a6c7f4-505d-421f-9823-8294a5a983c7} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.061269183 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:13.988537 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:13.989342 [ 3948 ] {3b553063-0a55-4a77-b2ec-eb50f51a166d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57260) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:13:13.998120 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_54_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:14.015026 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/zfb/mbeisodglxswzkzmkrjclqxhqsdrb, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 02a3578d-ed9b-4fec-b8f7-b2a4036377e0, upload has finished false.
2024.11.04 23:13:14.015147 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/zfb/mbeisodglxswzkzmkrjclqxhqsdrb, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 02a3578d-ed9b-4fec-b8f7-b2a4036377e0, upload has finished false
2024.11.04 23:13:14.051314 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/zfb/mbeisodglxswzkzmkrjclqxhqsdrb, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 02a3578d-ed9b-4fec-b8f7-b2a4036377e0, upload has finished false
2024.11.04 23:13:14.054754 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 42 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:14.082814 [ 3948 ] {3b553063-0a55-4a77-b2ec-eb50f51a166d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:13:15.383399 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:15.589132 [ 8080 ] {d368bf3b-54c2-4d19-bb63-b1e916b073d0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39444) (comment: 02842_largestTriangleThreeBuckets_aggregate_function.sql) (in query: select largestTriangleThreeBuckets(0)(x, y) FROM largestTriangleThreeBucketsTestFloat64Float64;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:15.594338 [ 8080 ] {d368bf3b-54c2-4d19-bb63-b1e916b073d0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:16.869081 [ 3885 ] {181e4531-6135-4f66-89e3-ef2c1ca4751d} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.055988891 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59706) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:16.869144 [ 3949 ] {b4bc7908-8357-4865-86c6-68b867e36c14} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016580185 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34888) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:16.869174 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:16.869661 [ 8080 ] {c3c70cec-4115-48da-a6cc-4679028053b1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55242) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:13:16.869668 [ 8081 ] {209f3748-d48e-4faf-ad22-c691386c0e0c} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.034317913 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51102) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:16.883403 [ 8080 ] {c3c70cec-4115-48da-a6cc-4679028053b1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:13:16.886714 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_55_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:16.893361 [ 3948 ] {98608dc8-1f44-49dd-851e-c5d3e3ef542d} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.155163999 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39440) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:16.904781 [ 8080 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:55242). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:16.936913 [ 3885 ] {181e4531-6135-4f66-89e3-ef2c1ca4751d} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.055988891 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:16.937829 [ 3885 ] {181e4531-6135-4f66-89e3-ef2c1ca4751d} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:16.970546 [ 8081 ] {209f3748-d48e-4faf-ad22-c691386c0e0c} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.034317913 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:16.971325 [ 8081 ] {209f3748-d48e-4faf-ad22-c691386c0e0c} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:17.010739 [ 3948 ] {98608dc8-1f44-49dd-851e-c5d3e3ef542d} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 2.155163999 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:17.809345 [ 8002 ] {} <Error> ServerErrorHandler: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:0: Poco::IOException::IOException(String const&, int) @ 0x000000003e9ff82e
2. ./build_docker/./base/poco/Net/src/NetException.cpp:0: Poco::Net::NetException::NetException(String const&, int) @ 0x000000003e668a8e
3. ./build_docker/./base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x000000003e685f7a
4. ./contrib/llvm-project/libcxx/include/string:1499: Poco::Net::SocketImpl::peerAddress() @ 0x000000003e68e6df
5. ./build_docker/./base/poco/Net/src/HTTPServerSession.cpp:65: Poco::Net::HTTPServerSession::clientAddress() @ 0x000000003e632a36
6. ./build_docker/./src/Server/HTTP/HTTPServerRequest.cpp:0: DB::HTTPServerRequest::HTTPServerRequest(std::shared_ptr<DB::IHTTPContext>, DB::HTTPServerResponse&, Poco::Net::HTTPServerSession&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a547c6f
7. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a545840
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:17.809408 [ 3949 ] {b4bc7908-8357-4865-86c6-68b867e36c14} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.016580185 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:17.809647 [ 8081 ] {53a666a7-7de8-49c5-ab86-d47910dc23f4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39446) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:13:17.809783 [ 3927 ] {643ccfd1-e267-4814-a320-edb077a6b90a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39454) (comment: 01710_projection_optimize_aggregators_of_group_by_keys.sql) (in query: CREATE TABLE proj (`date` Date, PROJECTION maxdate (SELECT max(date) GROUP BY date)) ENGINE = MergeTree ORDER BY tuple() SETTINGS index_granularity = 21087, min_bytes_for_wide_part = 974589105, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = false, max_file_name_length = 128, min_bytes_for_full_part_storage = 143717098, compact_parts_max_bytes_to_buffer = 51778559, compact_parts_max_granules_to_buffer = 231, compact_parts_merge_max_bytes_to_prefetch_part = 13012857, merge_max_block_size = 18038, old_parts_lifetime = 30., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 7307127606, index_granularity_bytes = 4037228, concurrent_part_removal_threshold = 6, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 75845, primary_key_compress_block_size = 28900 AS SELECT toDate('2012-10-24') - (number % 100) FROM numbers(100.)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:13:17.810116 [ 3949 ] {b4bc7908-8357-4865-86c6-68b867e36c14} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:17.812061 [ 8081 ] {53a666a7-7de8-49c5-ab86-d47910dc23f4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:13:17.814461 [ 3927 ] {643ccfd1-e267-4814-a320-edb077a6b90a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:13:18.804881 [ 4231 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:18.805522 [ 3163 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:18.805601 [ 3197 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:18.805682 [ 3202 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:18.805762 [ 3143 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:18.805966 [ 3204 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:18.806162 [ 3169 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:19.139294 [ 3196 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/async_blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:19.139353 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:19.141812 [ 3196 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
13. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
15. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
16. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:19.429641 [ 3948 ] {5d838365-e449-4f44-969c-5ecd214d673d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51118) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:19.440987 [ 3948 ] {5d838365-e449-4f44-969c-5ecd214d673d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:19.630989 [ 3949 ] {a5607ebc-3165-4238-a0f1-2844c3deb24c} <Error> executeQuery: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39474) (in query: select _path,_file from file('nonexist.txt', 'CSV', 'val1 char')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:13:19.686066 [ 3949 ] {a5607ebc-3165-4238-a0f1-2844c3deb24c} <Error> TCPHandler: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:13:19.832815 [ 8081 ] {e14adabf-c181-4087-b1ea-bd904e2b8adc} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.382893058 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39460) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:20.127868 [ 8081 ] {e14adabf-c181-4087-b1ea-bd904e2b8adc} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.382893058 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:20.127913 [ 8080 ] {b0f48105-8079-40ed-b9f5-9cbf0d8f8cba} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:34610). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34610) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:20.127997 [ 3885 ] {8e90f303-26c3-4c81-8df6-f31fb8ea4c71} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:55258). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55258) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:20.161926 [ 8080 ] {b0f48105-8079-40ed-b9f5-9cbf0d8f8cba} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:34610). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:20.162266 [ 8080 ] {b0f48105-8079-40ed-b9f5-9cbf0d8f8cba} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:34610). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:20.162879 [ 8080 ] {b0f48105-8079-40ed-b9f5-9cbf0d8f8cba} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:20.282102 [ 3885 ] {8e90f303-26c3-4c81-8df6-f31fb8ea4c71} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:55258). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:20.282416 [ 3885 ] {8e90f303-26c3-4c81-8df6-f31fb8ea4c71} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:55258). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:20.282907 [ 3885 ] {8e90f303-26c3-4c81-8df6-f31fb8ea4c71} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:20.834735 [ 8081 ] {f26963f3-0460-4fad-9002-69be5d465179} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39488) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:20.840519 [ 8081 ] {f26963f3-0460-4fad-9002-69be5d465179} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:22.809731 [ 9105 ] {5545904d-20c1-4bda-8675-647170d42b97} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:55266) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:22.814422 [ 9105 ] {5545904d-20c1-4bda-8675-647170d42b97} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:22.816796 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:22.876290 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/atm/xwrhiaxvrxazqqttfznltsxxerdud, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3c0e2f44-22aa-4bfd-9d18-2d5e86763373, upload has finished false.
2024.11.04 23:13:22.876411 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/atm/xwrhiaxvrxazqqttfznltsxxerdud, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3c0e2f44-22aa-4bfd-9d18-2d5e86763373, upload has finished false
2024.11.04 23:13:22.909265 [ 8081 ] {d7f17c8d-3473-4733-8a3f-182c4322f29b} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.41987034 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39518) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:22.935069 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/atm/xwrhiaxvrxazqqttfznltsxxerdud, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3c0e2f44-22aa-4bfd-9d18-2d5e86763373, upload has finished false
2024.11.04 23:13:22.942051 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 27 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:22.961322 [ 9105 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:55266). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:24.019390 [ 3885 ] {81c83ca5-d594-4e8a-90b4-53904933bc59} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:34904). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34904) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4fa7e8
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:24.019417 [ 8080 ] {2715bc34-6598-4ae1-8263-9d6573b371cd} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:37178). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37178) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:24.019566 [ 3948 ] {bdaa1d13-3fee-492a-8f60-02cf842a9e01} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:59718). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:59718) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:24.019671 [ 8081 ] {d7f17c8d-3473-4733-8a3f-182c4322f29b} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.41987034 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:24.019698 [ 9104 ] {245617bb-caba-45bf-939c-b09554780cfa} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:54756). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:54756) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:24.019700 [ 7099 ] {72b3d11a-87d8-4fb5-8f1c-cb8fe4b9e5c0} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:34618). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:34618) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:24.019715 [ 3949 ] {976fdb9b-1ad5-4b86-8a95-1048e5e090f0} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51122). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:51122) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:24.019787 [ 9108 ] {3592c2ac-6ceb-43f5-ac94-8e8cf0f5ac76} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:42308). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:42308) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.877275 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.877448 [ 3949 ] {976fdb9b-1ad5-4b86-8a95-1048e5e090f0} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51122). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.877540 [ 9108 ] {3592c2ac-6ceb-43f5-ac94-8e8cf0f5ac76} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:42308). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.877613 [ 9104 ] {245617bb-caba-45bf-939c-b09554780cfa} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:54756). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.877589 [ 7099 ] {72b3d11a-87d8-4fb5-8f1c-cb8fe4b9e5c0} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:34618). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.877768 [ 3949 ] {976fdb9b-1ad5-4b86-8a95-1048e5e090f0} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.5]:9000 -> [::ffff:127.0.0.1]:51122). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.877785 [ 3885 ] {81c83ca5-d594-4e8a-90b4-53904933bc59} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:34904). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.877837 [ 8080 ] {2715bc34-6598-4ae1-8263-9d6573b371cd} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:37178). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.877866 [ 3948 ] {bdaa1d13-3fee-492a-8f60-02cf842a9e01} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:59718). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:25.878058 [ 7099 ] {72b3d11a-87d8-4fb5-8f1c-cb8fe4b9e5c0} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.6]:9000 -> [::ffff:127.0.0.1]:34618). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.878135 [ 3885 ] {81c83ca5-d594-4e8a-90b4-53904933bc59} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.4]:9000 -> [::ffff:127.0.0.1]:34904). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4fa7e8
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.878116 [ 9108 ] {3592c2ac-6ceb-43f5-ac94-8e8cf0f5ac76} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.8]:9000 -> [::ffff:127.0.0.1]:42308). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.878056 [ 9104 ] {245617bb-caba-45bf-939c-b09554780cfa} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.7]:9000 -> [::ffff:127.0.0.1]:54756). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.878305 [ 8080 ] {2715bc34-6598-4ae1-8263-9d6573b371cd} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.9]:9000 -> [::ffff:127.0.0.1]:37178). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.878555 [ 3948 ] {bdaa1d13-3fee-492a-8f60-02cf842a9e01} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.3]:9000 -> [::ffff:127.0.0.1]:59718). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProgress() @ 0x000000003a4fbba7
5. ./contrib/llvm-project/libcxx/include/__mutex_base:143: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4faeff
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:13:25.879070 [ 3949 ] {976fdb9b-1ad5-4b86-8a95-1048e5e090f0} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:25.879199 [ 3885 ] {81c83ca5-d594-4e8a-90b4-53904933bc59} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:25.881975 [ 7099 ] {72b3d11a-87d8-4fb5-8f1c-cb8fe4b9e5c0} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:25.882061 [ 9104 ] {245617bb-caba-45bf-939c-b09554780cfa} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:25.882083 [ 9108 ] {3592c2ac-6ceb-43f5-ac94-8e8cf0f5ac76} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:25.882130 [ 8080 ] {2715bc34-6598-4ae1-8263-9d6573b371cd} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:25.882162 [ 3948 ] {bdaa1d13-3fee-492a-8f60-02cf842a9e01} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:25.963132 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 106 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:26.628487 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_55_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.326798 [ 9108 ] {95c8d3e9-b2b3-4e66-a98f-80b40d715812} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.015213524 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45222) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.326798 [ 7099 ] {ecde6deb-a7a1-4f8c-bd79-bbbd400777e0} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.034896689 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:44610) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.326822 [ 9132 ] {0853e708-9690-4d0b-8a6e-885446e6cf04} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.007364267 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:48146) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.326938 [ 3885 ] {ede15bc1-50e9-4f84-b460-2302d802a215} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:37682) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT `__table1`.`SearchPhrase` AS `k`, count() AS `c` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`SearchPhrase` ORDER BY count() DESC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationSparse.cpp:167: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cdd498
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:13:28.338599 [ 3885 ] {ede15bc1-50e9-4f84-b460-2302d802a215} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationSparse.cpp:167: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cdd498
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:13:28.343465 [ 8081 ] {73508822-838d-479d-a83a-f217b937d574} <Error> executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.95785483 seconds, maximum: 1. (TIMEOUT_EXCEEDED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33316) (comment: 00090_thread_pool_deadlock.sh) (in query: SELECT SearchPhrase AS k, count() AS c FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) GROUP BY k ORDER BY c DESC LIMIT 10), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.356670 [ 3885 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:37682). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.359169 [ 7099 ] {ecde6deb-a7a1-4f8c-bd79-bbbd400777e0} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.034896689 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.360097 [ 3948 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:43016). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.360427 [ 8080 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:51556). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.360529 [ 7099 ] {ecde6deb-a7a1-4f8c-bd79-bbbd400777e0} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:28.363645 [ 9131 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:58100). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.366487 [ 9104 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:52868). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.367152 [ 3949 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::ffff:127.0.0.1]:45478). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000001f50ad99
3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000001f50931e
4. DB::ReadBuffer::next() @ 0x0000000008ec3eef
5. ./src/IO/ReadBuffer.h:106: DB::TCPHandler::runImpl() @ 0x000000003a4d4e1a
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.395438 [ 8081 ] {73508822-838d-479d-a83a-f217b937d574} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.95785483 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.434057 [ 9132 ] {0853e708-9690-4d0b-8a6e-885446e6cf04} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.007364267 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.434618 [ 9132 ] {0853e708-9690-4d0b-8a6e-885446e6cf04} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:28.458225 [ 9108 ] {95c8d3e9-b2b3-4e66-a98f-80b40d715812} <Error> TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 1.015213524 seconds, maximum: 1. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<double, double>(int, FormatStringHelperImpl<std::type_identity<double>::type, std::type_identity<double>::type>, double&&, double&&) @ 0x00000000318cb8ec
3. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x00000000318caaa9
4. ./build_docker/./src/Interpreters/ProcessList.cpp:482: DB::QueryStatus::checkTimeLimit() @ 0x0000000035ab5ebb
5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000003a668b71
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a66842a
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850

2024.11.04 23:13:28.458749 [ 9108 ] {95c8d3e9-b2b3-4e66-a98f-80b40d715812} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:13:28.575092 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 32 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:28.726345 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_56_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:30.982570 [ 3757 ] {} <Error> void DB::SystemLog<DB::SessionLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::SessionLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::flushImpl(std::vector<DB::SessionLogElement, std::allocator<DB::SessionLogElement>> const&, unsigned long) @ 0x000000003604be4a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::savingThreadFunction() @ 0x00000000360485a8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f4369c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43689f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:30.982668 [ 8081 ] {44283bf4-8fda-45a9-be97-c10821b960d6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=35). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33320) (comment: 00087_where_0.sql) (in query: SELECT CounterID, uniq(UserID) FROM test.hits WHERE 0 != 0 GROUP BY CounterID;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:30.982771 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_60_12}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:30.995273 [ 8081 ] {44283bf4-8fda-45a9-be97-c10821b960d6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=35). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:31.001428 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_57_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:31.015053 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 13 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:32.575581 [ 8081 ] {0be57649-78ae-4701-b6ae-45a0707d549e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:33322) (comment: 00061_storage_buffer.sql) (in query: INSERT INTO test.hits_buffer SELECT * FROM test.hits WHERE CounterID = 800784;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:13:32.575623 [ 9105 ] {3ae3bb6b-0d3d-43b6-973a-950a4d40e4b9} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39528) (comment: 01560_crash_in_agg_empty_arglist.sql) (in query: SELECT any() as t, substring(query, 1, 70) AS query, avg(memory_usage) usage, count() count FROM system.query_log WHERE current_database = currentDatabase() AND event_date >= toDate(1604295323) AND event_time >= toDateTime(1604295323) AND type in (1,2,3,4) and initial_user in ('') and('all' = 'all' or(positionCaseInsensitive(query, 'all') = 1)) GROUP BY query ORDER BY usage desc LIMIT 5;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./src/AggregateFunctions/FactoryHelpers.h:27: DB::(anonymous namespace)::createAggregateFunctionAny(String const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::Array const&, DB::Settings const*) @ 0x000000002275d20a
4. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001fb1f7a3
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002fee7b68
6. ./build_docker/./src/AggregateFunctions/AggregateFunctionFactory.cpp:123: DB::AggregateFunctionFactory::get(String const&, DB::NullsAction, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::Array const&, DB::AggregateFunctionProperties&) const @ 0x000000002fee2d24
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5983: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034817c0d
8. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
19. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:13:32.577367 [ 9105 ] {3ae3bb6b-0d3d-43b6-973a-950a4d40e4b9} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./src/AggregateFunctions/FactoryHelpers.h:27: DB::(anonymous namespace)::createAggregateFunctionAny(String const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::Array const&, DB::Settings const*) @ 0x000000002275d20a
4. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001fb1f7a3
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002fee7b68
6. ./build_docker/./src/AggregateFunctions/AggregateFunctionFactory.cpp:123: DB::AggregateFunctionFactory::get(String const&, DB::NullsAction, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::Array const&, DB::AggregateFunctionProperties&) const @ 0x000000002fee2d24
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5983: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034817c0d
8. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
19. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:13:32.588154 [ 8081 ] {0be57649-78ae-4701-b6ae-45a0707d549e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:13:32.929899 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_57_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:35.737212 [ 3948 ] {9493e813-6691-43e6-83cd-da89685dfc5f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54508) (comment: 01518_select_in_null.sql) (in query: select count() from t1 where c1 in (select 'BBBBBB' union all select null);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:35.740505 [ 3948 ] {9493e813-6691-43e6-83cd-da89685dfc5f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:36.394400 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_57_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:36.810727 [ 8081 ] {5031b4ad-d9f8-459e-a8b7-f7e693513515} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54528) (comment: 00163_column_oriented_formats.sh) (in query: INSERT INTO 00163_column_oriented SELECT ClientEventTime, MobilePhoneModel, ClientIP6 FROM test.hits ORDER BY ClientEventTime, MobilePhoneModel, ClientIP6 LIMIT 100), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:13:36.839955 [ 8081 ] {5031b4ad-d9f8-459e-a8b7-f7e693513515} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:13:37.841575 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_57_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:38.304050 [ 3948 ] {e66f6635-067a-4d1e-9f2d-918669b2f37f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54552) (comment: 00163_column_oriented_formats.sh) (in query: SELECT MobilePhoneModel from 00163_column_oriented), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:38.307303 [ 3948 ] {e66f6635-067a-4d1e-9f2d-918669b2f37f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:40.538270 [ 3114 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:40.538382 [ 3101 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:40.538468 [ 3585 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:40.538604 [ 3113 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:40.538704 [ 3487 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:40.538835 [ 3121 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:40.540205 [ 3388 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Operation timeout, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/async_blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:40.880187 [ 3388 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
13. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
15. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
16. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:41.869278 [ 4839 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:41.869313 [ 8081 ] {72481df7-7b86-4350-be46-92650b368adf} <Error> executeQuery: Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/02981_insert_select_resize_to_max_insert_threads_test_13/02981_insert_select. (KEEPER_EXCEPTION) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54530) (comment: 02981_insert_select_resize_to_max_insert_threads.sh) (in query: CREATE TABLE t4_pr ( n UInt64, ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/02981_insert_select_resize_to_max_insert_threads_test_13/02981_insert_select', '1') ORDER BY n;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:384: zkutil::ZooKeeper::createAncestors(String const&) @ 0x000000003c03f674
5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::createTableIfNotExists(std::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0x000000003805a946
6. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions, bool) @ 0x0000000038051511
7. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageReplicatedMergeTree> std::allocate_shared[abi:v15000]<DB::StorageReplicatedMergeTree, std::allocator<DB::StorageReplicatedMergeTree>, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions&, bool&, void>(std::allocator<DB::StorageReplicatedMergeTree> const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, DB::RenamingRestrictions&, bool&) @ 0x0000000039d01af6
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2e7f
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
10. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
14. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:13:41.870775 [ 3348 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:41.870867 [ 3334 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:41.870896 [ 3336 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:41.871397 [ 3358 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:41.871574 [ 3362 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:41.871688 [ 3308 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:42.171739 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/bie/fkzxjlfzqbrvfxqeieksovzuexqya, total size 335544320, count 335544320, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 582bd210-fb4d-463c-88b6-b05996c6cb95, upload has finished false.
2024.11.04 23:13:42.171887 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/bie/fkzxjlfzqbrvfxqeieksovzuexqya, total size 335544320, count 335544320, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 582bd210-fb4d-463c-88b6-b05996c6cb95, upload has finished false
2024.11.04 23:13:42.221968 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/bie/fkzxjlfzqbrvfxqeieksovzuexqya, total size 335544320, count 335544320, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 582bd210-fb4d-463c-88b6-b05996c6cb95, upload has finished false
2024.11.04 23:13:42.484862 [ 8081 ] {72481df7-7b86-4350-be46-92650b368adf} <Error> TCPHandler: Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/02981_insert_select_resize_to_max_insert_threads_test_13/02981_insert_select. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:384: zkutil::ZooKeeper::createAncestors(String const&) @ 0x000000003c03f674
5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::createTableIfNotExists(std::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0x000000003805a946
6. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions, bool) @ 0x0000000038051511
7. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageReplicatedMergeTree> std::allocate_shared[abi:v15000]<DB::StorageReplicatedMergeTree, std::allocator<DB::StorageReplicatedMergeTree>, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, DB::RenamingRestrictions&, bool&, void>(std::allocator<DB::StorageReplicatedMergeTree> const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&, DB::RenamingRestrictions&, bool&) @ 0x0000000039d01af6
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2e7f
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
10. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
14. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:13:42.484837 [ 3794 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/task_queue/ddl/. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:384: zkutil::ZooKeeper::createAncestors(String const&) @ 0x000000003c03f674
5. ./build_docker/./src/Interpreters/DDLWorker.cpp:0: DB::DDLWorker::initializeMainThread() @ 0x0000000032b57b01
6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1141: DB::DDLWorker::runMainThread() @ 0x0000000032b0bb4d
7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x0000000032b5e0c7
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000032b5df1f
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:42.485100 [ 3344 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/async_blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:42.485263 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_65_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:42.485252 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_57_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:42.485354 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 2 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:42.485496 [ 3927 ] {1216838f-1088-4403-8792-df2592e5a1ab} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54610) (comment: 00163_column_oriented_formats.sh) (in query: INSERT INTO 00163_column_oriented SELECT ClientEventTime, MobilePhoneModel, ClientIP6 FROM test.hits ORDER BY ClientEventTime, MobilePhoneModel, ClientIP6 LIMIT 100), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:13:42.486608 [ 3344 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
13. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
15. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
16. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:42.693781 [ 3927 ] {1216838f-1088-4403-8792-df2592e5a1ab} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:13:44.144068 [ 8081 ] {084eaa80-9942-4099-be06-70e2bf975a55} <Error> executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 't4_pr' in scope SELECT * FROM t4_pr. (UNKNOWN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47568) (comment: 02981_insert_select_resize_to_max_insert_threads.sh) (in query:  EXPLAIN PIPELINE INSERT INTO t3_dist SELECT * FROM t4_pr; ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x000000000a10afd5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6970: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a48b5
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/vector:434: DB::InterpreterInsertQuery::execute() @ 0x00000000355ba6ce
9. ./src/QueryPipeline/QueryPipeline.h:135: DB::InterpreterExplainQuery::executeImpl() @ 0x00000000346cfe4d
10. ./build_docker/./src/Interpreters/InterpreterExplainQuery.cpp:0: DB::InterpreterExplainQuery::execute() @ 0x00000000346c9904
11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:13:44.144168 [ 3927 ] {8d4a517d-a770-471f-8d4c-a5907589ac08} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47572) (comment: 00163_column_oriented_formats.sh) (in query: SELECT MobilePhoneModel from 00163_column_oriented), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:44.146731 [ 8081 ] {084eaa80-9942-4099-be06-70e2bf975a55} <Error> TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 't4_pr' in scope SELECT * FROM t4_pr. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x000000000a10afd5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6970: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a48b5
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/vector:434: DB::InterpreterInsertQuery::execute() @ 0x00000000355ba6ce
9. ./src/QueryPipeline/QueryPipeline.h:135: DB::InterpreterExplainQuery::executeImpl() @ 0x00000000346cfe4d
10. ./build_docker/./src/Interpreters/InterpreterExplainQuery.cpp:0: DB::InterpreterExplainQuery::execute() @ 0x00000000346c9904
11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:13:44.147777 [ 3927 ] {8d4a517d-a770-471f-8d4c-a5907589ac08} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:44.166927 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_65_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=29). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:44.490684 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 6 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:44.585358 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_65_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:44.818321 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_57_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:44.998001 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_60_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:45.680638 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_65_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:45.998748 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_60_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:47.184640 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_65_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:47.667015 [ 8081 ] {92cfcafc-8962-4bb5-8548-5f572a61c6b1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47612) (comment: 00163_column_oriented_formats.sh) (in query: INSERT INTO 00163_column_oriented SELECT ClientEventTime, MobilePhoneModel, ClientIP6 FROM test.hits ORDER BY ClientEventTime, MobilePhoneModel, ClientIP6 LIMIT 100), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:13:47.698604 [ 8081 ] {92cfcafc-8962-4bb5-8548-5f572a61c6b1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:13:47.857849 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_60_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.464816 [ 3231 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:48.464953 [ 3226 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:48.464996 [ 3225 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:48.465127 [ 3203 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:48.465229 [ 3193 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:48.465293 [ 3201 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:13:48.465682 [ 3216 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.465865 [ 3212 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/1_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.466003 [ 3222 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.466521 [ 3217 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.466542 [ 3227 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.466585 [ 3219 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.466732 [ 3216 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.554791 [ 8081 ] {5358535a-1476-4ad1-ac62-abf96921d877} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47636) (comment: 00163_column_oriented_formats.sh) (in query: SELECT ClientEventTime from 00163_column_oriented), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:48.878946 [ 8081 ] {5358535a-1476-4ad1-ac62-abf96921d877} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:13:48.879012 [ 3974 ] {} <Error> ZooKeeperClient: Failed to connect to ZooKeeper: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=962). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c143bc3
7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:296: zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x000000003c031b77
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c037a9f
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:456: zkutil::ZooKeeper::startNewSession() const @ 0x000000003c06279c
10. ./build_docker/./src/Interpreters/Context.cpp:0: DB::Context::getZooKeeper() const @ 0x00000000329347c5
11. ./build_docker/./src/Interpreters/TransactionLog.cpp:0: DB::TransactionLog::runUpdatingThread() @ 0x000000003629a9cd
12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000362bc8a7
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000362bc6ff
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:48.882723 [ 3974 ] {} <Error> TransactionLog: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=962). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c143bc3
7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:296: zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x000000003c031b77
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c037a9f
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:456: zkutil::ZooKeeper::startNewSession() const @ 0x000000003c06279c
10. ./build_docker/./src/Interpreters/Context.cpp:0: DB::Context::getZooKeeper() const @ 0x00000000329347c5
11. ./build_docker/./src/Interpreters/TransactionLog.cpp:0: DB::TransactionLog::runUpdatingThread() @ 0x000000003629a9cd
12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000362bc8a7
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000362bc6ff
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:49.062584 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:50.190837 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_60_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:51.569135 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/niz/gsovcfnkolgzrfegzxaptnbuwhwwf, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 0e0b9c7f-b438-47b6-a450-8e4343d09000, upload has finished false.
2024.11.04 23:13:51.569315 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/niz/gsovcfnkolgzrfegzxaptnbuwhwwf, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 0e0b9c7f-b438-47b6-a450-8e4343d09000, upload has finished false
2024.11.04 23:13:51.606225 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/niz/gsovcfnkolgzrfegzxaptnbuwhwwf, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 0e0b9c7f-b438-47b6-a450-8e4343d09000, upload has finished false
2024.11.04 23:13:51.612258 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 31 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:52.070474 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_60_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:54.017300 [ 8081 ] {5af7b576-fda8-4dc0-aa51-a0c47c682dc1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47720) (comment: 03149_analyzer_window_redefinition.sql) (in query: INSERT INTO users VALUES ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86

2024.11.04 23:13:54.018662 [ 8081 ] {5af7b576-fda8-4dc0-aa51-a0c47c682dc1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86

2024.11.04 23:13:54.022387 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_61_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:54.051986 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 67 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:54.559458 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_61_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:57.540842 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_61_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:58.885868 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/who/uxtoaickrzgxghgnhrmdtgwhahyrk, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5a06c195-d46e-4f05-b065-b05c25b7588d, upload has finished false.
2024.11.04 23:13:58.886011 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/who/uxtoaickrzgxghgnhrmdtgwhahyrk, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5a06c195-d46e-4f05-b065-b05c25b7588d, upload has finished false
2024.11.04 23:13:58.897898 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/who/uxtoaickrzgxghgnhrmdtgwhahyrk, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5a06c195-d46e-4f05-b065-b05c25b7588d, upload has finished false
2024.11.04 23:13:59.294393 [ 8080 ] {} <Error> Access(user directories): from: ::ffff:127.0.0.1, user: default: Authentication failed: Code: 193. DB::Exception: Invalid credentials. (WRONG_PASSWORD), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Access/IAccessStorage.cpp:763: DB::IAccessStorage::throwInvalidCredentials() @ 0x000000003097182c
4. ./build_docker/./src/Access/IAccessStorage.cpp:0: DB::IAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970c32
5. ./build_docker/./src/Access/IAccessStorage.cpp:511: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970599
6. ./build_docker/./src/Access/MultipleAccessStorage.cpp:451: DB::MultipleAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030a365bb
7. ./build_docker/./src/Access/IAccessStorage.cpp:499: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool) const @ 0x00000000309703d1
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070f152
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
10. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:59.294643 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 5 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:13:59.294886 [ 3948 ] {80a938c3-3859-4c3d-908d-2944813b9397} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=25): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59570) (comment: 00172_hits_joins.gen.sql) (in query: SELECT EventDate, hits, visits FROM ( SELECT EventDate, count() AS hits FROM test.hits GROUP BY EventDate ) ANY LEFT JOIN ( SELECT StartDate AS EventDate, sum(Sign) AS visits FROM test.visits GROUP BY EventDate ) USING EventDate ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:00.044733 [ 3948 ] {80a938c3-3859-4c3d-908d-2944813b9397} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=25): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:00.044945 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_61_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:00.044976 [ 8080 ] {} <Error> ServerErrorHandler: Code: 516. DB::Exception: default: Authentication failed: password is incorrect, or there is no user with such name.

If you have installed ClickHouse and forgot password you can reset it in the configuration file.
The password for default user is typically located at /etc/clickhouse-server/users.d/default-password.xml
and deleting this file will reset the password.
See also /etc/clickhouse-server/users.xml on the server where ClickHouse is installed.

. (AUTHENTICATION_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:00.524170 [ 8081 ] {5c64a457-0ec7-4030-a8a4-3eda1b7bf458} <Error> executeQuery: Code: 516. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: default: Authentication failed: password is incorrect, or there is no user with such name.

If you have installed ClickHouse and forgot password you can reset it in the configuration file.
The password for default user is typically located at /etc/clickhouse-server/users.d/default-password.xml
and deleting this file will reset the password.
See also /etc/clickhouse-server/users.xml on the server where ClickHouse is installed.

. Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
. (AUTHENTICATION_FAILED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59572) (comment: 01119_session_log.sql) (in query: SELECT * FROM remote('127.0.0.2', system, one, 'default', '[HIDDEN]')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:452: DB::Connection::receiveHello(Poco::Timespan const&) @ 0x000000003a12d46c
6. ./build_docker/./src/Client/Connection.cpp:202: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x000000003a127a8a
7. ./build_docker/./src/Client/Connection.cpp:512: DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) @ 0x000000003a132b3e
8. ./build_docker/./src/Client/ConnectionEstablisher.cpp:0: DB::ConnectionEstablisher::run(PoolWithFailoverBase<DB::IConnectionPool>::TryResult&, String&) @ 0x000000003a15f2fc
9. ./build_docker/./src/Client/ConnectionEstablisher.cpp:133: DB::ConnectionEstablisherAsync::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a1654ac
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
12. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
13. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:00.525652 [ 8081 ] {5c64a457-0ec7-4030-a8a4-3eda1b7bf458} <Error> TCPHandler: Code: 516. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: default: Authentication failed: password is incorrect, or there is no user with such name.

If you have installed ClickHouse and forgot password you can reset it in the configuration file.
The password for default user is typically located at /etc/clickhouse-server/users.d/default-password.xml
and deleting this file will reset the password.
See also /etc/clickhouse-server/users.xml on the server where ClickHouse is installed.

. Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
. (AUTHENTICATION_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:452: DB::Connection::receiveHello(Poco::Timespan const&) @ 0x000000003a12d46c
6. ./build_docker/./src/Client/Connection.cpp:202: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x000000003a127a8a
7. ./build_docker/./src/Client/Connection.cpp:512: DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) @ 0x000000003a132b3e
8. ./build_docker/./src/Client/ConnectionEstablisher.cpp:0: DB::ConnectionEstablisher::run(PoolWithFailoverBase<DB::IConnectionPool>::TryResult&, String&) @ 0x000000003a15f2fc
9. ./build_docker/./src/Client/ConnectionEstablisher.cpp:133: DB::ConnectionEstablisherAsync::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a1654ac
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
12. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
13. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:01.420052 [ 8080 ] {} <Error> Access(user directories): from: ::ffff:127.0.0.1, user: nonexistsnt_user_1119: Authentication failed: Code: 192. DB::Exception: There is no user `nonexistsnt_user_1119` in local_directory. (UNKNOWN_USER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String const&>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String const&>::type>, String&&, String const&) @ 0x000000000a065755
3. ./build_docker/./src/Access/IAccessStorage.cpp:694: DB::IAccessStorage::throwNotFound(DB::AccessEntityType, String const&) const @ 0x00000000309651a3
4. ./build_docker/./src/Access/IAccessStorage.cpp:529: DB::IAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970c11
5. ./build_docker/./src/Access/IAccessStorage.cpp:511: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970599
6. ./build_docker/./src/Access/MultipleAccessStorage.cpp:451: DB::MultipleAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030a365bb
7. ./build_docker/./src/Access/IAccessStorage.cpp:499: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool) const @ 0x00000000309703d1
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070f152
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
10. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:01.420567 [ 3948 ] {4c789d5f-4fc3-4e75-980d-3a84055b7ece} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59582) (comment: 00047_bar.sql) (in query: SELECT CounterID, count() AS c, bar(c, 0, 523264) FROM test.hits GROUP BY CounterID ORDER BY c DESC, CounterID ASC LIMIT 100 SETTINGS optimize_aggregation_in_order = 1 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:01.421039 [ 8080 ] {} <Error> ServerErrorHandler: Code: 516. DB::Exception: nonexistsnt_user_1119: Authentication failed: password is incorrect, or there is no user with such name. (AUTHENTICATION_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:01.425236 [ 8081 ] {78dfd0dc-c7ce-45c3-88aa-717d0d760a29} <Error> executeQuery: Code: 516. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: nonexistsnt_user_1119: Authentication failed: password is incorrect, or there is no user with such name.. Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
. (AUTHENTICATION_FAILED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59572) (comment: 01119_session_log.sql) (in query: SELECT * FROM remote('127.0.0.2', system, one, 'nonexistsnt_user_1119', '[HIDDEN]')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:452: DB::Connection::receiveHello(Poco::Timespan const&) @ 0x000000003a12d46c
6. ./build_docker/./src/Client/Connection.cpp:202: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x000000003a127a8a
7. ./build_docker/./src/Client/Connection.cpp:512: DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) @ 0x000000003a132b3e
8. ./build_docker/./src/Client/ConnectionEstablisher.cpp:0: DB::ConnectionEstablisher::run(PoolWithFailoverBase<DB::IConnectionPool>::TryResult&, String&) @ 0x000000003a15f2fc
9. ./build_docker/./src/Client/ConnectionEstablisher.cpp:133: DB::ConnectionEstablisherAsync::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a1654ac
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
12. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
13. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:01.426564 [ 8081 ] {78dfd0dc-c7ce-45c3-88aa-717d0d760a29} <Error> TCPHandler: Code: 516. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: nonexistsnt_user_1119: Authentication failed: password is incorrect, or there is no user with such name.. Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
. (AUTHENTICATION_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:452: DB::Connection::receiveHello(Poco::Timespan const&) @ 0x000000003a12d46c
6. ./build_docker/./src/Client/Connection.cpp:202: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x000000003a127a8a
7. ./build_docker/./src/Client/Connection.cpp:512: DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) @ 0x000000003a132b3e
8. ./build_docker/./src/Client/ConnectionEstablisher.cpp:0: DB::ConnectionEstablisher::run(PoolWithFailoverBase<DB::IConnectionPool>::TryResult&, String&) @ 0x000000003a15f2fc
9. ./build_docker/./src/Client/ConnectionEstablisher.cpp:133: DB::ConnectionEstablisherAsync::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a1654ac
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
12. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
13. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:01.484788 [ 3948 ] {4c789d5f-4fc3-4e75-980d-3a84055b7ece} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:03.594852 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:03.595093 [ 3948 ] {13feef8e-408a-4d24-8a70-ea92b8b4e317} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59588) (comment: 00169_contingency.sql) (in query: WITH URLDomain AS a, URLDomain AS b SELECT round(cramersV(a, b), 2), round(cramersVBiasCorrected(a, b), 2), round(theilsU(a, b), 2), round(theilsU(b, a), 2), round(contingency(a, b), 2) FROM test.hits;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:03.609028 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_61_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:03.629501 [ 3948 ] {13feef8e-408a-4d24-8a70-ea92b8b4e317} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:04.979743 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/dnd/ydvqxgspmbeksjpdebpztgfdszmzt, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f504e762-11e4-4638-9a44-25a56e568ffa, upload has finished false.
2024.11.04 23:14:04.979875 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/dnd/ydvqxgspmbeksjpdebpztgfdszmzt, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f504e762-11e4-4638-9a44-25a56e568ffa, upload has finished false
2024.11.04 23:14:05.010809 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/dnd/ydvqxgspmbeksjpdebpztgfdszmzt, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f504e762-11e4-4638-9a44-25a56e568ffa, upload has finished false
2024.11.04 23:14:05.015715 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 60 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:05.331957 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_62_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:06.067755 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_62_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:07.888638 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:09.331695 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/oat/nwiazmpfrqojnayhyjzbgxazxrclm, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id eb178b44-9f32-465f-b5a5-24b5d0f16120, upload has finished false.
2024.11.04 23:14:09.331857 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/oat/nwiazmpfrqojnayhyjzbgxazxrclm, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id eb178b44-9f32-465f-b5a5-24b5d0f16120, upload has finished false
2024.11.04 23:14:09.338349 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/oat/nwiazmpfrqojnayhyjzbgxazxrclm, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id eb178b44-9f32-465f-b5a5-24b5d0f16120, upload has finished false
2024.11.04 23:14:09.342264 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 26 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:10.939645 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_62_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:11.760714 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_63_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:13.394637 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:13.396397 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_100_125_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:13.434273 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 2 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:14.282197 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:14.500022 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_100_125_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:14.914967 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:15.315558 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_100_125_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:18.042317 [ 3519 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:18.042857 [ 3533 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:18.042880 [ 3516 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:18.043057 [ 3520 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:18.043064 [ 3515 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:18.043051 [ 3523 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:18.044511 [ 3447 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:18.044808 [ 3510 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:18.044860 [ 3527 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:18.048543 [ 3508 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:18.051078 [ 3480 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:18.426686 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/yza/dscutadtirwolbladdiejayssoalo, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 7bfac325-7e77-4404-b1f4-44f84baee1c1, upload has finished false.
2024.11.04 23:14:18.426932 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/yza/dscutadtirwolbladdiejayssoalo, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 7bfac325-7e77-4404-b1f4-44f84baee1c1, upload has finished false
2024.11.04 23:14:18.438658 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/yza/dscutadtirwolbladdiejayssoalo, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 7bfac325-7e77-4404-b1f4-44f84baee1c1, upload has finished false
2024.11.04 23:14:18.444593 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 49 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:19.019065 [ 3292 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:19.019259 [ 3314 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:19.019283 [ 3310 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:19.019513 [ 3313 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:19.019506 [ 3287 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:19.019942 [ 3311 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:19.096315 [ 3692 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded hard memory limit (14.54 GiB)
2024.11.04 23:14:19.096820 [ 3692 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded soft memory limit (13.77 GiB)
2024.11.04 23:14:19.896433 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:19.896515 [ 3324 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:19.896584 [ 3282 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:19.896639 [ 3332 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:20.639904 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 2 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:20.657124 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_63_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:21.461818 [ 3398 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:21.461950 [ 3282 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:21.462030 [ 3324 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:21.462160 [ 3332 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:21.462552 [ 3367 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:21.462672 [ 3378 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:21.464769 [ 3336 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:22.481342 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 67 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:23.380792 [ 3228 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:23.380900 [ 3219 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:23.381035 [ 3233 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:23.381203 [ 3234 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:23.381557 [ 3216 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:23.381875 [ 3230 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:23.382471 [ 3227 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:23.382590 [ 3236 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:23.384158 [ 3227 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:23.467845 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_64_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:25.698042 [ 3948 ] {f34c5451-55e5-4472-880b-d21e796ea224} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50106) (comment: 00084_external_aggregation.sql) (in query: SELECT URL, uniq(SearchPhrase) AS u FROM test.hits GROUP BY URL ORDER BY u DESC, URL LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:14:25.930430 [ 3948 ] {f34c5451-55e5-4472-880b-d21e796ea224} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:14:26.333723 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_71_14}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:26.786841 [ 7099 ] {1e6c96b5-be29-4fe5-8488-998c76607c47} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:50006) (comment: 00173_group_by_use_nulls.sql) (in query: SELECT `__table1`.`CounterID` AS `k`, quantileBFloat16(0.5)(`__table1`.`ResolutionWidth`) AS `quantileBFloat16(0.5)(ResolutionWidth)` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`CounterID` ORDER BY count() DESC, `__table1`.`CounterID` ASC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:26.955645 [ 7099 ] {1e6c96b5-be29-4fe5-8488-998c76607c47} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:26.991715 [ 3948 ] {a616bc41-743d-40c2-bd54-ff2936f3a434} <Error> executeQuery: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
: While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52746) (comment: 00173_group_by_use_nulls.sql) (in query: SELECT CounterID AS k, quantileBFloat16(0.5)(ResolutionWidth) FROM remote('127.0.0.{1,2}', test, hits) GROUP BY k ORDER BY count() DESC, CounterID ASC LIMIT 10 SETTINGS group_by_use_nulls = 1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:27.116798 [ 3948 ] {a616bc41-743d-40c2-bd54-ff2936f3a434} <Error> TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
: While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:28.267636 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:28.267741 [ 8080 ] {e7a508cb-3585-415a-ac61-603fa5cd9a76} <Error> executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:57580). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:57580) (comment: 00173_group_by_use_nulls.sql) (in query: SELECT `__table1`.`CounterID` AS `k`, quantileBFloat16(0.5)(`__table1`.`ResolutionWidth`) AS `quantileBFloat16(0.5)(ResolutionWidth)` FROM `test`.`hits` AS `__table1` GROUP BY `__table1`.`CounterID` ORDER BY count() DESC, `__table1`.`CounterID` ASC LIMIT _CAST(10, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4fa7e8
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:14:28.267853 [ 3948 ] {4e6f3fc7-669c-4f5e-9ba2-679d64ce2c43} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52760) (comment: 00041_aggregating_materialized_view.sql) (in query: SELECT StartDate, sumMerge(Visits) AS Visits, uniqMerge(Users) AS Users FROM test.basic GROUP BY StartDate ORDER BY StartDate;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:28.284315 [ 3948 ] {4e6f3fc7-669c-4f5e-9ba2-679d64ce2c43} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:28.362245 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_64_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:28.472655 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/cor/jgleqsxplgrybcgcvoeabjnxtzyfh, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b9f7965f-c822-4d49-a4b4-e75f76eb1f23, upload has finished false.
2024.11.04 23:14:28.472805 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/cor/jgleqsxplgrybcgcvoeabjnxtzyfh, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b9f7965f-c822-4d49-a4b4-e75f76eb1f23, upload has finished false
2024.11.04 23:14:28.495275 [ 8080 ] {e7a508cb-3585-415a-ac61-603fa5cd9a76} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:57580). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:28.495572 [ 8080 ] {e7a508cb-3585-415a-ac61-603fa5cd9a76} <Error> TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:57580). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50edb9
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000003a501fc7
5. ./src/Core/Block.h:37: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4fa7e8
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:14:28.496049 [ 8080 ] {e7a508cb-3585-415a-ac61-603fa5cd9a76} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:14:28.502813 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/cor/jgleqsxplgrybcgcvoeabjnxtzyfh, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b9f7965f-c822-4d49-a4b4-e75f76eb1f23, upload has finished false
2024.11.04 23:14:28.507815 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 14 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:29.234383 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_65_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:30.131577 [ 3948 ] {2c74e29b-1542-44e9-8bde-2525753ca408} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52762) (comment: 00094_order_by_array_join_limit.sql) (in query: SELECT `ParsedParams.Key2` AS x FROM test.hits ARRAY JOIN ParsedParams AS PP ORDER BY x ASC LIMIT 2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:30.189954 [ 3948 ] {2c74e29b-1542-44e9-8bde-2525753ca408} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:32.159751 [ 3948 ] {8fc6de56-5abc-470d-bda4-dcabb10b9c07} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52772) (comment: 00062_loyalty.sql) (in query: SELECT loyalty, count() AS c, bar(log(c + 1) * 1000, 0, log(6000) * 1000, 80) FROM (SELECT UserID, toInt8((yandex > google ? yandex / (yandex + google) : -google / (yandex + google)) * 10) AS loyalty FROM (SELECT UserID, sum(SearchEngineID = 2) AS yandex, sum(SearchEngineID = 3) AS google FROM test.hits WHERE SearchEngineID = 2 OR SearchEngineID = 3 GROUP BY UserID HAVING yandex + google > 10)) GROUP BY loyalty ORDER BY loyalty), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:32.159791 [ 8081 ] {0a6220b7-f1cc-455b-b756-a26304894348} <Error> executeQuery: Code: 519. DB::NetException: All attempts to get table structure failed. Log: 

Code: 279. DB::NetException: All connection tries failed. Log: 

Timeout exceeded while reading from socket (socket (127.0.0.2:9000), receive timeout 10000 ms)
Timeout exceeded while reading from socket (socket (127.0.0.2:9000), receive timeout 10000 ms)
Timeout exceeded while reading from socket (socket (127.0.0.2:9000), receive timeout 10000 ms)

. (ALL_CONNECTION_TRIES_FAILED) (version 24.3.12.76.altinitystable (altinity build))

. (NO_REMOTE_SHARD_AVAILABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59572) (comment: 01119_session_log.sql) (in query: SELECT * FROM remote('127.0.0.2', system, one, ' INTERSERVER SECRET ', '[HIDDEN]')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x000000003874696a
3. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:169: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x0000000038740132
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
5. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a91a9
6. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
7. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
12. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
17. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
18. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
19. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
21. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
25. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
26. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850

2024.11.04 23:14:32.161335 [ 8081 ] {0a6220b7-f1cc-455b-b756-a26304894348} <Error> TCPHandler: Code: 519. DB::NetException: All attempts to get table structure failed. Log: 

Code: 279. DB::NetException: All connection tries failed. Log: 

Timeout exceeded while reading from socket (socket (127.0.0.2:9000), receive timeout 10000 ms)
Timeout exceeded while reading from socket (socket (127.0.0.2:9000), receive timeout 10000 ms)
Timeout exceeded while reading from socket (socket (127.0.0.2:9000), receive timeout 10000 ms)

. (ALL_CONNECTION_TRIES_FAILED) (version 24.3.12.76.altinitystable (altinity build))

. (NO_REMOTE_SHARD_AVAILABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x000000003874696a
3. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:169: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x0000000038740132
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
5. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::executeImpl(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool) const @ 0x000000002f4a91a9
6. ./build_docker/./src/TableFunctions/ITableFunction.cpp:0: DB::ITableFunction::execute(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>, String const&, DB::ColumnsDescription, bool, bool) const @ 0x000000002fefdd15
7. ./build_docker/./src/Interpreters/Context.cpp:2014: DB::Context::executeTableFunction(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::ITableFunction> const&) @ 0x000000003290cdaf
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveTableFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&, bool) @ 0x00000000347cc59f
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d4ae5
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
12. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
17. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
18. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
19. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
21. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
25. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
26. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850

2024.11.04 23:14:32.275448 [ 3948 ] {8fc6de56-5abc-470d-bda4-dcabb10b9c07} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:32.276817 [ 8080 ] {} <Error> Access(user directories): from: ::ffff:127.0.0.1, user:    : Authentication failed: Code: 192. DB::Exception: There is no user `   ` in local_directory. (UNKNOWN_USER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String const&>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String const&>::type>, String&&, String const&) @ 0x000000000a065755
3. ./build_docker/./src/Access/IAccessStorage.cpp:694: DB::IAccessStorage::throwNotFound(DB::AccessEntityType, String const&) const @ 0x00000000309651a3
4. ./build_docker/./src/Access/IAccessStorage.cpp:529: DB::IAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970c11
5. ./build_docker/./src/Access/IAccessStorage.cpp:511: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970599
6. ./build_docker/./src/Access/MultipleAccessStorage.cpp:451: DB::MultipleAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030a365bb
7. ./build_docker/./src/Access/IAccessStorage.cpp:499: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool) const @ 0x00000000309703d1
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070f152
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
10. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:32.277713 [ 8080 ] {} <Error> ServerErrorHandler: Code: 516. DB::Exception:    : Authentication failed: password is incorrect, or there is no user with such name. (AUTHENTICATION_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:32.282909 [ 8081 ] {c17a3099-00b6-4169-9d8c-349ce5b064d1} <Error> executeQuery: Code: 516. DB::Exception: Received from 127.0.0.2:9000. DB::Exception:    : Authentication failed: password is incorrect, or there is no user with such name.. Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
. (AUTHENTICATION_FAILED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59572) (comment: 01119_session_log.sql) (in query: SELECT * FROM remote('127.0.0.2', system, one, '   ', '[HIDDEN]')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:452: DB::Connection::receiveHello(Poco::Timespan const&) @ 0x000000003a12d46c
6. ./build_docker/./src/Client/Connection.cpp:202: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x000000003a127a8a
7. ./build_docker/./src/Client/Connection.cpp:512: DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) @ 0x000000003a132b3e
8. ./build_docker/./src/Client/ConnectionEstablisher.cpp:0: DB::ConnectionEstablisher::run(PoolWithFailoverBase<DB::IConnectionPool>::TryResult&, String&) @ 0x000000003a15f2fc
9. ./build_docker/./src/Client/ConnectionEstablisher.cpp:133: DB::ConnectionEstablisherAsync::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a1654ac
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
12. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
13. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:32.298197 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_65_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:32.358925 [ 8081 ] {c17a3099-00b6-4169-9d8c-349ce5b064d1} <Error> TCPHandler: Code: 516. DB::Exception: Received from 127.0.0.2:9000. DB::Exception:    : Authentication failed: password is incorrect, or there is no user with such name.. Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./src/Access/Credentials.h:51: DB::Session::authenticate(String const&, String const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3a666
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receiveHello() @ 0x000000003a4ec97c
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d3de4
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
. (AUTHENTICATION_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:452: DB::Connection::receiveHello(Poco::Timespan const&) @ 0x000000003a12d46c
6. ./build_docker/./src/Client/Connection.cpp:202: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x000000003a127a8a
7. ./build_docker/./src/Client/Connection.cpp:512: DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) @ 0x000000003a132b3e
8. ./build_docker/./src/Client/ConnectionEstablisher.cpp:0: DB::ConnectionEstablisher::run(PoolWithFailoverBase<DB::IConnectionPool>::TryResult&, String&) @ 0x000000003a15f2fc
9. ./build_docker/./src/Client/ConnectionEstablisher.cpp:133: DB::ConnectionEstablisherAsync::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a1654ac
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
12. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
13. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:33.357698 [ 8002 ] {5d0ebf82-589a-4efb-9d28-c63f899f7073} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:41236) (in query: select 1 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::finalizeImpl() @ 0x000000003abfdfd1
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c1ddb
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:14:33.369180 [ 8002 ] {5d0ebf82-589a-4efb-9d28-c63f899f7073} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::finalizeImpl() @ 0x000000003abfdfd1
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c1ddb
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:33.577347 [ 8081 ] {87cd90f6-e6ba-4e64-9b63-29e5d2041fbe} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59572) (comment: 01119_session_log.sql) (in query: select * from url('http://127.0.0.1:8123/?query=select+1&user=default', LineAsString, 's String');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:33.584988 [ 8081 ] {87cd90f6-e6ba-4e64-9b63-29e5d2041fbe} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:34.691972 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_65_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:35.342006 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/wta/zdykwerdczzpwvmkgvepzyeisasxc, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2ade107c-cbf4-4de2-a512-c9aa07029a20, upload has finished false.
2024.11.04 23:14:35.342235 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/wta/zdykwerdczzpwvmkgvepzyeisasxc, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2ade107c-cbf4-4de2-a512-c9aa07029a20, upload has finished false
2024.11.04 23:14:35.382905 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/wta/zdykwerdczzpwvmkgvepzyeisasxc, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2ade107c-cbf4-4de2-a512-c9aa07029a20, upload has finished false
2024.11.04 23:14:35.388440 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:35.514860 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_65_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:36.113820 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 11 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:36.881832 [ 8080 ] {ad376f92-6934-4c28-93c3-fe53d2bf0c4a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:45444) (comment: 00088_global_in_one_shard_and_rows_before_limit.sql) (in query: SELECT `__table1`.`EventDate` AS `EventDate`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`UserID` GLOBAL IN (`_data_10265056313896149572_17723635565405517704` AS `__table2`) GROUP BY `__table1`.`EventDate` ORDER BY `__table1`.`EventDate` ASC LIMIT _CAST(5, 'UInt64')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:36.943643 [ 8080 ] {ad376f92-6934-4c28-93c3-fe53d2bf0c4a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:36.950685 [ 3948 ] {0ced0dd9-7021-4e34-a4a7-0e6a8091d31f} <Error> executeQuery: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
: While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52784) (comment: 00088_global_in_one_shard_and_rows_before_limit.sql) (in query: SELECT EventDate, count() FROM remote('127.0.0.1', test.hits) WHERE UserID GLOBAL IN (SELECT UserID FROM test.hits) GROUP BY EventDate ORDER BY EventDate LIMIT 5 FORMAT JSONCompact;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:36.974295 [ 3948 ] {0ced0dd9-7021-4e34-a4a7-0e6a8091d31f} <Error> TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace:

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
: While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000001f251da8
3. ./src/IO/WriteBufferFromVector.h:65: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001f25b5a6
4. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000003a131f5f
5. ./build_docker/./src/Client/Connection.cpp:1070: DB::Connection::receivePacket() @ 0x000000003a14e351
6. ./build_docker/./src/Client/PacketReceiver.cpp:66: DB::PacketReceiver::Task::run(std::function<void (int, Poco::Timespan, DB::AsyncEventTimeoutType, String const&, unsigned int)>, std::function<void ()>) @ 0x000000003a21f9d0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191da7c
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003191d222
9. ./contrib/boost/boost/context/fiber_ucontext.hpp:0: boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>::run() @ 0x000000003191ca33
10. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func<boost::context::detail::fiber_capture_record<boost::context::fiber, FiberStack&, Fiber::RoutineImpl<DB::AsyncTaskExecutor::Routine>>>(unsigned int, unsigned int) @ 0x000000003191a052

2024.11.04 23:14:37.396647 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_70_14}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:38.967620 [ 3948 ] {8c96e4c5-dc16-4885-b58e-e8a3342600bb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58426) (comment: 00074_full_join.sql) (in query: SELECT CounterID, hits, visits FROM ( SELECT (CounterID % 100000) AS CounterID, count() AS hits FROM test.hits GROUP BY CounterID ) ANY FULL OUTER JOIN ( SELECT (CounterID % 100000) AS CounterID, sum(Sign) AS visits FROM test.visits GROUP BY CounterID HAVING visits > 0 ) USING CounterID WHERE hits = 0 OR visits = 0 ORDER BY hits + visits * 10 DESC, CounterID ASC LIMIT 20;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:39.008318 [ 3948 ] {8c96e4c5-dc16-4885-b58e-e8a3342600bb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:39.852978 [ 3948 ] {3ce60c8e-e5a2-40c8-b4a8-75ec26ab439f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58486) (comment: 00144_functions_of_aggregation_states.sql) (in query: SELECT EventDate, finalizeAggregation(state), runningAccumulate(state) FROM (SELECT EventDate, uniqState(UserID) AS state FROM test.hits GROUP BY EventDate ORDER BY EventDate);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:39.864586 [ 3948 ] {3ce60c8e-e5a2-40c8-b4a8-75ec26ab439f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:40.761180 [ 8080 ] {2a37867b-ac2a-4765-b9ef-2671e183ed5a} <Error> executeQuery: Code: 344. DB::Exception: Experimental Inverted Index feature is not enabled (turn on setting 'allow_experimental_inverted_index'). (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58490) (comment: 02346_inverted_index_experimental_flag.sql) (in query: ALTER TABLE tab ADD INDEX inv_idx(str) TYPE inverted(0);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3124: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000003917979c
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:14:40.780663 [ 8080 ] {2a37867b-ac2a-4765-b9ef-2671e183ed5a} <Error> TCPHandler: Code: 344. DB::Exception: Experimental Inverted Index feature is not enabled (turn on setting 'allow_experimental_inverted_index'). (SUPPORT_IS_DISABLED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3124: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000003917979c
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:14:41.253155 [ 3550 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:41.253249 [ 3551 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:41.253346 [ 3547 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:41.253460 [ 3519 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:41.253553 [ 3523 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:41.254012 [ 3584 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:41.259474 [ 3572 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:41.276505 [ 3549 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
12. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
15. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
17. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
18. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:41.292472 [ 3556 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:44.635566 [ 3949 ] {6de6711d-32a5-4445-a45c-d29d3dc1efe2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=964). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58524) (comment: 02586_generate_random_structure.sql) (in query: select toTypeName(generateRandomStructure(5, 42));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:14:44.635787 [ 3948 ] {b23cf644-adf9-4aed-9fa1-a74c3aa8af56} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58498) (comment: 00037_uniq_state_merge1.sql) (in query: SELECT k, any(u) AS u, uniqMerge(us) AS us FROM (SELECT domain(URL) AS k, uniq(UserID) AS u, uniqState(UserID) AS us FROM test.hits GROUP BY k) GROUP BY k ORDER BY u DESC, k ASC LIMIT 100 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b780475
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:44.811163 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/twg/sigyqkxygxdglvjuuofpwtekcvape, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8244ccb9-9287-486a-97d1-d1bd6134f21a, upload has finished false.
2024.11.04 23:14:44.811365 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/twg/sigyqkxygxdglvjuuofpwtekcvape, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8244ccb9-9287-486a-97d1-d1bd6134f21a, upload has finished false
2024.11.04 23:14:44.927548 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/twg/sigyqkxygxdglvjuuofpwtekcvape, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8244ccb9-9287-486a-97d1-d1bd6134f21a, upload has finished false
2024.11.04 23:14:46.229096 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:46.229114 [ 3949 ] {6de6711d-32a5-4445-a45c-d29d3dc1efe2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=964). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:14:46.229360 [ 3948 ] {b23cf644-adf9-4aed-9fa1-a74c3aa8af56} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b780475
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:46.229454 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 43 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:46.229454 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_65_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:46.637809 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:46.875747 [ 3948 ] {71dfb3f1-68a0-4be5-bf2b-e4c5e5d50ab3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=964). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46704) (comment: 02751_parallel_replicas_bug_chunkinfo_not_set.sql) (in query: INSERT INTO join_inner_table__fuzz_1 SELECT CAST('833c9e22-c245-4eb5-8745-117a9a1f26b1', 'UUID') AS id, CAST(rowNumberInAllBlocks(), 'String') AS key, * FROM generateRandom('number Int64, value1 String, value2 String, time Int64', 1, 10, 2) LIMIT 100;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:14:46.880112 [ 3948 ] {71dfb3f1-68a0-4be5-bf2b-e4c5e5d50ab3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=964). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:14:47.283391 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:47.814548 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:47.858922 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:48.820514 [ 3949 ] {579507dc-5adf-4ccd-a56c-5666117ce91a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46714) (comment: 00054_merge_tree_partitions.sql) (in query: INSERT INTO test.partitions SELECT EventDate + UserID % 365 AS EventDate, CounterID FROM test.hits WHERE CounterID = 1704509;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:14:48.887468 [ 3949 ] {579507dc-5adf-4ccd-a56c-5666117ce91a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:14:49.190203 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:50.334062 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:51.063976 [ 3949 ] {3dec4ff4-a0e9-484a-970b-74ef4f9edce1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46736) (comment: 00035_aggregate_keys128.sql) (in query: SELECT SearchEngineID AS k1, AdvEngineID AS k2, count() AS c FROM test.hits GROUP BY k1, k2 ORDER BY c DESC, k1, k2 LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:51.303576 [ 3949 ] {3dec4ff4-a0e9-484a-970b-74ef4f9edce1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:14:53.536713 [ 3949 ] {ec333c16-c426-4e9f-925f-6b43497a30bc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46806) (comment: 00152_insert_different_granularity.sql) (in query: INSERT INTO fixed_granularity_table SELECT * FROM test.hits LIMIT 10;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:14:53.575219 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:53.833360 [ 3949 ] {ec333c16-c426-4e9f-925f-6b43497a30bc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:14:54.381133 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:54.389419 [ 3948 ] {f3bc74a0-65ed-4f73-b030-1e878fe99efc} <Error> executeQuery: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37342) (in query: select _path,_file from file('nonexist.txt', 'CSV', 'val1 char')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:14:54.390943 [ 3948 ] {f3bc74a0-65ed-4f73-b030-1e878fe99efc} <Error> TCPHandler: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:14:55.101263 [ 3503 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:55.101555 [ 3514 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:55.101572 [ 3507 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:55.101622 [ 3511 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:55.101724 [ 3429 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:55.101798 [ 3513 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:14:55.903003 [ 3299 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Operation timeout, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:55.903088 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:57.325664 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:57.325754 [ 3299 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
13. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
15. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
16. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:57.325777 [ 3948 ] {e2268496-8136-466f-b776-3be3e0f79712} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37398) (comment: 00060_move_to_prewhere_and_sets.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE TraficSourceID IN (7);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:57.349981 [ 3948 ] {e2268496-8136-466f-b776-3be3e0f79712} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:14:58.126668 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:14:59.399706 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_100_136_7}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:00.097840 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_67_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:01.127864 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/laj/glfpqfjvmzucoldtosrulthmbodew, total size 469762048, count 469762048, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 68d46fca-df62-4289-b3de-3e7b55be26b9, upload has finished false.
2024.11.04 23:15:01.128040 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/laj/glfpqfjvmzucoldtosrulthmbodew, total size 469762048, count 469762048, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 68d46fca-df62-4289-b3de-3e7b55be26b9, upload has finished false
2024.11.04 23:15:01.207315 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/laj/glfpqfjvmzucoldtosrulthmbodew, total size 469762048, count 469762048, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 68d46fca-df62-4289-b3de-3e7b55be26b9, upload has finished false
2024.11.04 23:15:01.214818 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_8_8_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 91 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:04.041953 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_69_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:05.050605 [ 3948 ] {0dae0d78-2f9f-4d77-b1cf-c6b90900b4cc} <Error> executeQuery: Code: 33. DB::Exception: Last saved offsset for File a.txt is bigger than file size (311 > 0). (CANNOT_READ_ALL_DATA) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44490) (comment: 02023_storage_filelog.sh) (in query: select * from file_log order by k settings stream_like_engine_allow_direct_select=1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long&, std::fpos<__mbstate_t>&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long&>::type, std::type_identity<std::fpos<__mbstate_t>&>::type>, String const&, unsigned long&, std::fpos<__mbstate_t>&) @ 0x00000000388349da
3. ./build_docker/./src/Storages/FileLog/StorageFileLog.cpp:479: DB::StorageFileLog::openFilesAndSetPos() @ 0x0000000038815897
4. ./build_docker/./src/Storages/FileLog/StorageFileLog.cpp:103: DB::ReadFromStorageFileLog::makePipe() @ 0x0000000038841045
5. ./src/QueryPipeline/Pipe.h:108: DB::ReadFromStreamLikeEngine::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b8468d9
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:15:05.050876 [ 8080 ] {5f551ec7-5d5b-437c-ac9f-31faee052401} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44488) (comment: 00056_view.sql) (in query: SELECT count() FROM test.view;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:15:05.053000 [ 3948 ] {0dae0d78-2f9f-4d77-b1cf-c6b90900b4cc} <Error> TCPHandler: Code: 33. DB::Exception: Last saved offsset for File a.txt is bigger than file size (311 > 0). (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long&, std::fpos<__mbstate_t>&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long&>::type, std::type_identity<std::fpos<__mbstate_t>&>::type>, String const&, unsigned long&, std::fpos<__mbstate_t>&) @ 0x00000000388349da
3. ./build_docker/./src/Storages/FileLog/StorageFileLog.cpp:479: DB::StorageFileLog::openFilesAndSetPos() @ 0x0000000038815897
4. ./build_docker/./src/Storages/FileLog/StorageFileLog.cpp:103: DB::ReadFromStorageFileLog::makePipe() @ 0x0000000038841045
5. ./src/QueryPipeline/Pipe.h:108: DB::ReadFromStreamLikeEngine::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b8468d9
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:15:05.082368 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_69_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:05.202843 [ 8080 ] {5f551ec7-5d5b-437c-ac9f-31faee052401} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
27. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
28. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b

2024.11.04 23:15:05.204769 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:06.174718 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/gfp/zcadwsrohjyiibtrkguyptfchsofd, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d68b5528-5f79-402e-bc31-45887bbeebb4, upload has finished false.
2024.11.04 23:15:06.174888 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/gfp/zcadwsrohjyiibtrkguyptfchsofd, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d68b5528-5f79-402e-bc31-45887bbeebb4, upload has finished false
2024.11.04 23:15:06.187458 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/gfp/zcadwsrohjyiibtrkguyptfchsofd, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d68b5528-5f79-402e-bc31-45887bbeebb4, upload has finished false
2024.11.04 23:15:06.191556 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 6 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:06.851731 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_76_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:07.516618 [ 8080 ] {cf45adf0-ac20-4936-82c9-e173be967d00} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44500) (comment: 00032_aggregate_key64.sql) (in query: SELECT SearchEngineID AS k1, count() AS c FROM test.hits GROUP BY k1 ORDER BY c DESC, k1 LIMIT 10 ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:07.806230 [ 8080 ] {cf45adf0-ac20-4936-82c9-e173be967d00} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:08.024653 [ 3948 ] {9e884a7b-b81f-4a4e-ab5b-ab60029866ff} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44504) (comment: 00578_merge_table_and_table_virtual_column.sql) (in query: SELECT DISTINCT count() FROM merge(currentDatabase(), '^numbers\\d+$') GROUP BY number;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:08.043643 [ 3948 ] {9e884a7b-b81f-4a4e-ab5b-ab60029866ff} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:08.179480 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 79 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:08.786529 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_69_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./build_docker/./src/Storages/MergeTree/MergedColumnOnlyOutputStream.cpp:0: DB::MergedColumnOnlyOutputStream::fillChecksums(std::shared_ptr<DB::IMergeTreeDataPart>&, DB::MergeTreeDataPartChecksums&) @ 0x000000003989e2cd
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:658: DB::MergeTask::VerticalMergeStage::finalizeVerticalMergeForOneColumn() const @ 0x000000003905fc0d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b276
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:10.247838 [ 3948 ] {abae8826-c541-425d-94d6-6dd4d3752f27} <Error> virtual void DB::StorageBuffer::flushAndPrepareForShutdown(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x00000000387d34a8
9. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAndPrepareForShutdown() @ 0x00000000387d2e4f
10. ./src/Storages/IStorage.h:580: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr<DB::Context const>, DB::ASTDropQuery&, std::shared_ptr<DB::IDatabase>&, StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag>&) @ 0x00000000346b6ed3
11. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000346b39bc
12. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:73: DB::InterpreterDropQuery::execute() @ 0x00000000346b0cfe
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:10.248149 [ 8080 ] {eb5a092d-90b2-4c88-b242-2e4da60b6bf0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=33, jobs=33). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44512) (comment: 01671_aggregate_function_group_bitmap_data.sql) (in query: SELECT bitmapCardinality(day_today) AS today_users, bitmapCardinality(day_before) AS before_users, bitmapCardinality(bitmapOr(day_today, day_before)) AS ll_users, bitmapCardinality(bitmapAnd(day_today, day_before)) AS old_users, bitmapCardinality(bitmapAndnot(day_today, day_before)) AS new_users, bitmapCardinality(bitmapXor(day_today, day_before)) AS diff_users FROM ( SELECT city_id, groupBitmapState(uid) AS day_today FROM group_bitmap_data_test WHERE pickup_date = '2019-01-02' GROUP BY uid, city_id ) AS js1 ALL LEFT JOIN ( SELECT city_id, groupBitmapState(uid) AS day_before FROM group_bitmap_data_test WHERE pickup_date = '2019-01-01' GROUP BY city_id ) AS js2 USING (city_id) ORDER BY today_users, before_users, ll_users, old_users, new_users, diff_users;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:10.850536 [ 3097 ] {} <Error> void DB::StorageBuffer::backgroundFlush(): Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:1050: DB::StorageBuffer::backgroundFlush() @ 0x00000000387e1f16
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000387e667f
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:10.850597 [ 8080 ] {eb5a092d-90b2-4c88-b242-2e4da60b6bf0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=33, jobs=33). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:11.902009 [ 3948 ] {c01b1138-9f66-4b3c-aeb4-4011de61f2ef} <Error> executeQuery: Code: 47. DB::Exception: Unknown expression identifier 'nonexistentsomething' in scope SELECT nonexistentsomething. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44508) (comment: 01083_expressions_in_engine_arguments.sql) (in query: -- The following line is needed just to disable checking stderr for emptiness
 SELECT nonexistentsomething;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String&, String const&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String const&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String const&, String&&, String&&) @ 0x0000000034910cc2
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6420: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347c14e0
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:15:11.902069 [ 3757 ] {} <Error> void DB::SystemLog<DB::SessionLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::SessionLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::flushImpl(std::vector<DB::SessionLogElement, std::allocator<DB::SessionLogElement>> const&, unsigned long) @ 0x000000003604be4a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::savingThreadFunction() @ 0x00000000360485a8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f4369c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43689f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:11.903323 [ 3948 ] {c01b1138-9f66-4b3c-aeb4-4011de61f2ef} <Error> TCPHandler: Code: 47. DB::Exception: Unknown expression identifier 'nonexistentsomething' in scope SELECT nonexistentsomething. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String&, String const&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String const&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String const&, String&&, String&&) @ 0x0000000034910cc2
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6420: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347c14e0
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:15:11.914890 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_70_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:12.199760 [ 3738 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:15:12.199905 [ 3738 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:15:13.202307 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/sov/hbqoizvkvwdajqszhagjtmtpimprs, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d1ef7c15-e0a0-41ff-a06f-1506b1b48203, upload has finished false.
2024.11.04 23:15:13.202470 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/sov/hbqoizvkvwdajqszhagjtmtpimprs, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d1ef7c15-e0a0-41ff-a06f-1506b1b48203, upload has finished false
2024.11.04 23:15:13.222791 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/sov/hbqoizvkvwdajqszhagjtmtpimprs, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d1ef7c15-e0a0-41ff-a06f-1506b1b48203, upload has finished false
2024.11.04 23:15:14.883720 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:15.808048 [ 8080 ] {b6e9ee5e-ca0c-4431-a0b8-4a386488910d} <Error> executeQuery: Code: 48. DB::Exception: JOIN ON constant supported only with join algorithm 'hash'. (NOT_IMPLEMENTED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44514) (comment: 02461_join_lc_issue_42380.sql) (in query: SELECT * FROM t1__fuzz_13 FULL OUTER JOIN t2__fuzz_47 ON 1 = 2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ab79
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:15:15.808212 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 41 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:15.810228 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f35a2ef5-e44d-4e42-94b5-c841517c6ce9::202411_1_66_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:15.810823 [ 8080 ] {b6e9ee5e-ca0c-4431-a0b8-4a386488910d} <Error> TCPHandler: Code: 48. DB::Exception: JOIN ON constant supported only with join algorithm 'hash'. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ab79
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:15:16.684984 [ 8080 ] {61d5a088-6e47-4d55-b2c0-05da7565d999} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48040) (comment: 02417_json_object_each_row_format.sql) (in query: select * from file(02417_data.jsonObjectEachRow);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:16.689285 [ 8080 ] {61d5a088-6e47-4d55-b2c0-05da7565d999} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:17.325159 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:17.990699 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_70_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:19.200086 [ 8080 ] {82858a6d-ae98-4358-bfe4-8acdf3efbcdd} <Error> executeQuery: Code: 223. DB::Exception: All elements of Enum data type must be of form: 'name' = number or 'name', where name is string literal and number is an integer. (UNEXPECTED_AST_STRUCTURE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48054) (comment: 00757_enum_defaults.sql) (in query: CREATE TABLE auto_assign_enum2 (`x` enum('a' = -1000, 'b', 'c' = -99)) ENGINE = MergeTree ORDER BY x SETTINGS index_granularity = 7790, min_bytes_for_wide_part = 1073741824, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = false, max_file_name_length = 128, min_bytes_for_full_part_storage = 536870912, compact_parts_max_bytes_to_buffer = 22115977, compact_parts_max_granules_to_buffer = 106, compact_parts_merge_max_bytes_to_prefetch_part = 19779346, merge_max_block_size = 7793, old_parts_lifetime = 246., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 10, min_merge_bytes_to_use_direct_io = 1, index_granularity_bytes = 2868050, concurrent_part_removal_threshold = 100, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = true, marks_compress_block_size = 21827, primary_key_compress_block_size = 62670), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:236: DB::autoAssignNumberForEnum(std::shared_ptr<DB::IAST> const&) @ 0x000000003194e2de
4. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:0: DB::create(std::shared_ptr<DB::IAST> const&) @ 0x000000003194bb38
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003193221a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003196b4bb
7. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:93: std::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031969e41
8. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:74: DB::DataTypeFactory::get(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031963537
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:539: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x0000000034617def
10. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:15:19.966990 [ 3948 ] {ab25665d-6095-4c70-8298-2cfbe057cbec} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:15:21.022919 [ 8080 ] {82858a6d-ae98-4358-bfe4-8acdf3efbcdd} <Error> TCPHandler: Code: 223. DB::Exception: All elements of Enum data type must be of form: 'name' = number or 'name', where name is string literal and number is an integer. (UNEXPECTED_AST_STRUCTURE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:236: DB::autoAssignNumberForEnum(std::shared_ptr<DB::IAST> const&) @ 0x000000003194e2de
4. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:0: DB::create(std::shared_ptr<DB::IAST> const&) @ 0x000000003194bb38
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003193221a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003196b4bb
7. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:93: std::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031969e41
8. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:74: DB::DataTypeFactory::get(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031963537
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:539: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x0000000034617def
10. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:15:21.022951 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:21.023190 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 7 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:21.023270 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_70_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:22.385605 [ 3948 ] {ab25665d-6095-4c70-8298-2cfbe057cbec} <Error> Cluster: Code: 198. DB::NetException: Not found address of host: localhosx. (DNS_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x000000001f07d0ea
3. ./build_docker/./src/Common/DNSResolver.cpp:112: DB::(anonymous namespace)::hostByName(String const&) @ 0x000000001f0660d2
4. ./build_docker/./src/Common/DNSResolver.cpp:0: DB::(anonymous namespace)::resolveIPAddressImpl(String const&) @ 0x000000001f0615f2
5. ./build_docker/./src/Common/DNSResolver.cpp:0: DB::(anonymous namespace)::resolveIPAddressWithCache(DB::CacheBase<String, DB::DNSResolver::CacheEntry, std::hash<String>, DB::EqualWeightFunction<DB::DNSResolver::CacheEntry>>&, String const&) @ 0x000000001f062c3e
6. ./build_docker/./src/Common/DNSResolver.cpp:170: DB::DNSResolver::resolveAddress(String const&, unsigned short) @ 0x000000001f064eb4
7. ./build_docker/./src/Interpreters/Cluster.cpp:0: DB::Cluster::Address::Address(DB::DatabaseReplicaInfo const&, DB::ClusterConnectionParameters const&, unsigned int, unsigned int) @ 0x000000003288ed54
8. ./contrib/llvm-project/libcxx/include/vector:1580: void std::vector<DB::Cluster::Address, std::allocator<DB::Cluster::Address>>::__emplace_back_slow_path<DB::DatabaseReplicaInfo, DB::ClusterConnectionParameters const&, unsigned int&, unsigned long>(DB::DatabaseReplicaInfo&&, DB::ClusterConnectionParameters const&, unsigned int&, unsigned long&&) @ 0x00000000328c320d
9. ./contrib/llvm-project/libcxx/include/vector:0: DB::Cluster::Cluster(DB::Settings const&, std::vector<std::vector<String, std::allocator<String>>, std::allocator<std::vector<String, std::allocator<String>>>> const&, DB::ClusterConnectionParameters const&) @ 0x00000000328ac532
10. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::Cluster> std::make_shared[abi:v15000]<DB::Cluster, DB::Settings const&, std::vector<std::vector<String, std::allocator<String>>, std::allocator<std::vector<String, std::allocator<String>>>>&, DB::ClusterConnectionParameters&, void>(DB::Settings const&, std::vector<std::vector<String, std::allocator<String>>, std::allocator<std::vector<String, std::allocator<String>>>>&, DB::ClusterConnectionParameters&) @ 0x000000002f4af78a
11. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a3b6c
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionFactory::get(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) const @ 0x000000002ff12886
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:823: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x0000000034624a89
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:22.453954 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:23.919930 [ 8080 ] {20f67fa8-ee1e-4c92-af75-13675428609c} <Error> executeQuery: Code: 223. DB::Exception: Elements of Enum data type must be of form: 'name' = number or 'name', where name is string literal and number is an integer. (UNEXPECTED_AST_STRUCTURE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48054) (comment: 00757_enum_defaults.sql) (in query: CREATE TABLE auto_assign_enum3 (`x` enum('a', 'b', NULL)) ENGINE = MergeTree ORDER BY x SETTINGS index_granularity = 7790, min_bytes_for_wide_part = 1073741824, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = false, max_file_name_length = 128, min_bytes_for_full_part_storage = 536870912, compact_parts_max_bytes_to_buffer = 22115977, compact_parts_max_granules_to_buffer = 106, compact_parts_merge_max_bytes_to_prefetch_part = 19779346, merge_max_block_size = 7793, old_parts_lifetime = 246., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 10, min_merge_bytes_to_use_direct_io = 1, index_granularity_bytes = 2868050, concurrent_part_removal_threshold = 100, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = true, marks_compress_block_size = 21827, primary_key_compress_block_size = 62670), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:268: std::shared_ptr<DB::IDataType const> DB::createExact<DB::DataTypeEnum<_BitInt(8)>>(DB::IDataType<DB::IAST> const&) @ 0x000000003194a2e0
4. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:0: DB::create(std::shared_ptr<DB::IAST> const&) @ 0x000000003194bc25
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003193221a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003196b4bb
7. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:93: std::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031969e41
8. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:74: DB::DataTypeFactory::get(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031963537
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:539: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x0000000034617def
10. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:15:23.921510 [ 8080 ] {20f67fa8-ee1e-4c92-af75-13675428609c} <Error> TCPHandler: Code: 223. DB::Exception: Elements of Enum data type must be of form: 'name' = number or 'name', where name is string literal and number is an integer. (UNEXPECTED_AST_STRUCTURE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:268: std::shared_ptr<DB::IDataType const> DB::createExact<DB::DataTypeEnum<_BitInt(8)>>(DB::IDataType<DB::IAST> const&) @ 0x000000003194a2e0
4. ./build_docker/./src/DataTypes/DataTypeEnum.cpp:0: DB::create(std::shared_ptr<DB::IAST> const&) @ 0x000000003194bc25
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003193221a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003196b4bb
7. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:93: std::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031969e41
8. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:74: DB::DataTypeFactory::get(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031963537
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:539: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x0000000034617def
10. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:15:23.928554 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_70_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:24.272008 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:24.516565 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_75_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:24.638396 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_70_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:25.938116 [ 8080 ] {bbb98e95-7469-4cbe-b1c0-cb1453f881fc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58536) (comment: 02313_avro_records_and_maps.sql) (in query: select * from file(data_02313.avro);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:25.939285 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_75_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:25.941147 [ 8080 ] {bbb98e95-7469-4cbe-b1c0-cb1453f881fc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:25.944816 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_70_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:26.234460 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_71_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:27.211643 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_75_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=36). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:27.534294 [ 8080 ] {3d4ba00f-fd15-412c-bb0b-170a417701fc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=965). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58540) (comment: 01721_join_implicit_cast_long.gen.sql) (in query: SELECT '= right =';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:15:27.535580 [ 8080 ] {3d4ba00f-fd15-412c-bb0b-170a417701fc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=965). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:15:27.825885 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/yfz/mfbywcpndpbkhnkzzmsmnjrketbce, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1fc47761-d929-42f4-a831-2cf13b54fb2e, upload has finished false.
2024.11.04 23:15:27.826078 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/yfz/mfbywcpndpbkhnkzzmsmnjrketbce, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1fc47761-d929-42f4-a831-2cf13b54fb2e, upload has finished false
2024.11.04 23:15:27.846008 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/yfz/mfbywcpndpbkhnkzzmsmnjrketbce, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1fc47761-d929-42f4-a831-2cf13b54fb2e, upload has finished false
2024.11.04 23:15:27.942147 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 78 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:27.974398 [ 3238 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:15:30.389820 [ 3948 ] {ab25665d-6095-4c70-8298-2cfbe057cbec} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:15:30.431798 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:31.907285 [ 3948 ] {ab25665d-6095-4c70-8298-2cfbe057cbec} <Error> Cluster: Code: 198. DB::NetException: Not found address of host: localhosy. (DNS_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x000000001f07d0ea
3. ./build_docker/./src/Common/DNSResolver.cpp:112: DB::(anonymous namespace)::hostByName(String const&) @ 0x000000001f0660d2
4. ./build_docker/./src/Common/DNSResolver.cpp:0: DB::(anonymous namespace)::resolveIPAddressImpl(String const&) @ 0x000000001f0615f2
5. ./build_docker/./src/Common/DNSResolver.cpp:0: DB::(anonymous namespace)::resolveIPAddressWithCache(DB::CacheBase<String, DB::DNSResolver::CacheEntry, std::hash<String>, DB::EqualWeightFunction<DB::DNSResolver::CacheEntry>>&, String const&) @ 0x000000001f062c3e
6. ./build_docker/./src/Common/DNSResolver.cpp:170: DB::DNSResolver::resolveAddress(String const&, unsigned short) @ 0x000000001f064eb4
7. ./build_docker/./src/Interpreters/Cluster.cpp:0: DB::Cluster::Address::Address(DB::DatabaseReplicaInfo const&, DB::ClusterConnectionParameters const&, unsigned int, unsigned int) @ 0x000000003288ed54
8. ./contrib/llvm-project/libcxx/include/vector:1580: void std::vector<DB::Cluster::Address, std::allocator<DB::Cluster::Address>>::__emplace_back_slow_path<DB::DatabaseReplicaInfo, DB::ClusterConnectionParameters const&, unsigned int&, unsigned long>(DB::DatabaseReplicaInfo&&, DB::ClusterConnectionParameters const&, unsigned int&, unsigned long&&) @ 0x00000000328c320d
9. ./contrib/llvm-project/libcxx/include/vector:0: DB::Cluster::Cluster(DB::Settings const&, std::vector<std::vector<String, std::allocator<String>>, std::allocator<std::vector<String, std::allocator<String>>>> const&, DB::ClusterConnectionParameters const&) @ 0x00000000328ac532
10. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::Cluster> std::make_shared[abi:v15000]<DB::Cluster, DB::Settings const&, std::vector<std::vector<String, std::allocator<String>>, std::allocator<std::vector<String, std::allocator<String>>>>&, DB::ClusterConnectionParameters&, void>(DB::Settings const&, std::vector<std::vector<String, std::allocator<String>>, std::allocator<std::vector<String, std::allocator<String>>>>&, DB::ClusterConnectionParameters&) @ 0x000000002f4af78a
11. ./build_docker/./src/TableFunctions/TableFunctionRemote.cpp:0: DB::TableFunctionRemote::parseArguments(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) @ 0x000000002f4a3b6c
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionFactory::get(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const>) const @ 0x000000002ff12886
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:823: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x0000000034624a89
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:31.907302 [ 8080 ] {0334ae55-5c16-47ed-8f8c-08de991dad02} <Error> executeQuery: Code: 10. DB::Exception: Column `s` not found in table test_7.t1__fuzz_13 (d67eee96-21ff-4a07-a48e-e1eab3112d59). (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58550) (comment: 01551_context_uaf.sql) (in query: select * from merge('', 'f');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageSnapshot.cpp:214: DB::StorageSnapshot::getSampleBlockForColumns(std::vector<String, std::allocator<String>> const&) const @ 0x000000003865fde4
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::ReadFromMergeTree(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>>, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const> const&, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<Poco::Logger>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) @ 0x000000003b69f2ba
5. ./contrib/llvm-project/libcxx/include/vector:434: std::__unique_if<DB::ReadFromMergeTree>::__unique_single std::make_unique[abi:v15000]<DB::ReadFromMergeTree, std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&>(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&) @ 0x000000003946b1ef
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:921: DB::MergeTreeDataSelectExecutor::readFromParts(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) const @ 0x000000003942bebf
7. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:140: DB::MergeTreeDataSelectExecutor::read(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, bool) const @ 0x000000003942b247
8. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000039d0d26b
9. ./src/Processors/QueryPlan/QueryPlan.h:54: DB::ReadFromMerge::createPlanForTable(std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, DB::QueryProcessingStage::Enum, unsigned long, std::tuple<String, std::shared_ptr<DB::IStorage>, std::shared_ptr<DB::RWLockImpl::LockHolderImpl>, String> const&, std::vector<String, std::allocator<String>>&&, std::optional<DB::ReadFromMerge::RowPolicyData> const&, std::shared_ptr<DB::Context>, unsigned long) const @ 0x0000000037f6fb8f
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: DB::ReadFromMerge::createChildrenPlans(DB::SelectQueryInfo&) const @ 0x0000000037f5bbeb
11. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadFromMerge::filterTablesAndCreateChildrenPlans() @ 0x0000000037f4ad18
12. ./build_docker/./src/Storages/StorageMerge.cpp:1576: DB::ReadFromMerge::applyFilters(DB::ActionDAGNodes) @ 0x0000000037f87a1d
13. ./src/Processors/QueryPlan/SourceStepWithFilter.h:0: DB::QueryPlanOptimizations::optimizePrimaryKeyCondition(std::vector<DB::QueryPlanOptimizations::Frame, std::allocator<DB::QueryPlanOptimizations::Frame>> const&) @ 0x000000003b8f5b49
14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:0: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list<DB::QueryPlan::Node, std::allocator<DB::QueryPlan::Node>>&) @ 0x000000003b8ab2c2
15. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000003b66ac55
16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:0: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b667910
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:15:31.907505 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:31.907579 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_75_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:31.910578 [ 8080 ] {0334ae55-5c16-47ed-8f8c-08de991dad02} <Error> TCPHandler: Code: 10. DB::Exception: Column `s` not found in table test_7.t1__fuzz_13 (d67eee96-21ff-4a07-a48e-e1eab3112d59). (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageSnapshot.cpp:214: DB::StorageSnapshot::getSampleBlockForColumns(std::vector<String, std::allocator<String>> const&) const @ 0x000000003865fde4
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::ReadFromMergeTree(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>>, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const> const&, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<Poco::Logger>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) @ 0x000000003b69f2ba
5. ./contrib/llvm-project/libcxx/include/vector:434: std::__unique_if<DB::ReadFromMergeTree>::__unique_single std::make_unique[abi:v15000]<DB::ReadFromMergeTree, std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&>(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::vector<String, std::allocator<String>> const&, DB::MergeTreeData const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::Context const>&, unsigned long const&, unsigned long const&, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>&, std::shared_ptr<Poco::Logger> const&, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>&, bool&) @ 0x000000003946b1ef
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:921: DB::MergeTreeDataSelectExecutor::readFromParts(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, std::shared_ptr<DB::ReadFromMergeTree::AnalysisResult>, bool) const @ 0x000000003942bebf
7. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:140: DB::MergeTreeDataSelectExecutor::read(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, bool) const @ 0x000000003942b247
8. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000039d0d26b
9. ./src/Processors/QueryPlan/QueryPlan.h:54: DB::ReadFromMerge::createPlanForTable(std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, DB::QueryProcessingStage::Enum, unsigned long, std::tuple<String, std::shared_ptr<DB::IStorage>, std::shared_ptr<DB::RWLockImpl::LockHolderImpl>, String> const&, std::vector<String, std::allocator<String>>&&, std::optional<DB::ReadFromMerge::RowPolicyData> const&, std::shared_ptr<DB::Context>, unsigned long) const @ 0x0000000037f6fb8f
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: DB::ReadFromMerge::createChildrenPlans(DB::SelectQueryInfo&) const @ 0x0000000037f5bbeb
11. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadFromMerge::filterTablesAndCreateChildrenPlans() @ 0x0000000037f4ad18
12. ./build_docker/./src/Storages/StorageMerge.cpp:1576: DB::ReadFromMerge::applyFilters(DB::ActionDAGNodes) @ 0x0000000037f87a1d
13. ./src/Processors/QueryPlan/SourceStepWithFilter.h:0: DB::QueryPlanOptimizations::optimizePrimaryKeyCondition(std::vector<DB::QueryPlanOptimizations::Frame, std::allocator<DB::QueryPlanOptimizations::Frame>> const&) @ 0x000000003b8f5b49
14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:0: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list<DB::QueryPlan::Node, std::allocator<DB::QueryPlan::Node>>&) @ 0x000000003b8ab2c2
15. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000003b66ac55
16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:0: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b667910
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
20. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
22. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
26. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
27. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:15:31.925915 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_71_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:32.001659 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:33.640093 [ 3948 ] {ab25665d-6095-4c70-8298-2cfbe057cbec} <Error> executeQuery: Code: 81. DB::Exception: Database FAIL does not exist. (UNKNOWN_DATABASE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44508) (comment: 01083_expressions_in_engine_arguments.sql) (in query: -- dict --> url --> merge |-> distributed -> file (1)
 --                        |-> distributed_tf -> buffer -> file (1)
 -- TODO make fuzz test from this
 CREATE TABLE rich_syntax as remote ( 'localhos{x|y|t}', cluster ( 'test' || '_' || 'shard_localhost', remote ( '127.0.0.{1..4}', if ( toString(40 + 2) NOT IN ('hello', dictGetString(currentDatabase() || '.dict', 'col', toUInt64('0001'))), currentDatabase(), 'FAIL' ), extract('123view456', '[a-z]+') ) ) );), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b78581
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
5. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:53: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x00000000387355f7
6. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
8. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:42: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x0000000038735487
9. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
11. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:42: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x0000000038735487
12. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
14. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x0000000034624e15
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
16. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
17. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
18. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
20. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
21. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
23. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
24. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
25. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
26. ? @ 0x00007fe94d0adac3
27. ? @ 0x00007fe94d13f850

2024.11.04 23:15:33.642133 [ 3948 ] {ab25665d-6095-4c70-8298-2cfbe057cbec} <Error> TCPHandler: Code: 81. DB::Exception: Database FAIL does not exist. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000032b78581
4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:953: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000032b8ae03
5. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:53: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x00000000387355f7
6. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
8. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:42: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x0000000038735487
9. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
11. ./build_docker/./src/Storages/getStructureOfRemoteTable.cpp:42: DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x0000000038735487
12. ./contrib/boost/boost/multi_index_container.hpp:677: DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, std::shared_ptr<DB::Context const>, std::shared_ptr<DB::IAST> const&) @ 0x000000003873f8ed
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:596: DB::TableFunctionRemote::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4ab5b9
14. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x0000000034624e15
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
16. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
17. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
18. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
20. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
21. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
23. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
24. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
25. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
26. ? @ 0x00007fe94d0adac3
27. ? @ 0x00007fe94d13f850

2024.11.04 23:15:33.719963 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:35.105823 [ 7099 ] {f84c7868-b237-437f-a817-84f81caa32b0} <Error> executeQuery: Code: 42. DB::Exception: Array data type family must have exactly one argument - type of elements. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46924) (comment: 00507_array_no_params.sh) (in query: CREATE TABLE foo (a Array) Engine=Memory;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/DataTypes/DataTypeArray.cpp:81: DB::create(std::shared_ptr<DB::IAST> const&) @ 0x0000000031934393
4. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003193221a
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003196b4bb
6. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:93: std::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031969e41
7. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:74: DB::DataTypeFactory::get(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031963537
8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:539: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x0000000034617def
9. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:15:35.108321 [ 7099 ] {f84c7868-b237-437f-a817-84f81caa32b0} <Error> TCPHandler: Code: 42. DB::Exception: Array data type family must have exactly one argument - type of elements. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/DataTypes/DataTypeArray.cpp:81: DB::create(std::shared_ptr<DB::IAST> const&) @ 0x0000000031934393
4. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003193221a
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003196b4bb
6. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:93: std::shared_ptr<DB::IDataType const> DB::DataTypeFactory::getImpl<false>(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031969e41
7. ./build_docker/./src/DataTypes/DataTypeFactory.cpp:74: DB::DataTypeFactory::get(std::shared_ptr<DB::IAST> const&) const @ 0x0000000031963537
8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:539: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x0000000034617def
9. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
11. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:15:35.436352 [ 3948 ] {faea0c26-805d-4222-b2eb-d620b4d0c0f0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46936) (comment: 00597_with_totals_on_empty_set.sql) (in query: SELECT count() FROM numbers(10) WHERE number = -1 WITH TOTALS FORMAT Vertical;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:35.439473 [ 3948 ] {faea0c26-805d-4222-b2eb-d620b4d0c0f0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:35.653113 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 67 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:36.668420 [ 3948 ] {0444a7ec-c68e-4c87-b33a-0d75ec7b1c1e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46940) (comment: 01746_executable_pool_dictionary.sql) (in query: SELECT dictGet('executable_pool_complex', 'a', ('First_1', 'Second_1'));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:15:36.670501 [ 3948 ] {0444a7ec-c68e-4c87-b33a-0d75ec7b1c1e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:15:37.697503 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_71_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:38.384280 [ 3948 ] {ef9fb212-7c00-48ad-83bb-05370be2d2fe} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP key s column which is a part of key expression. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46948) (comment: 01716_drop_rename_sign_column.sql) (in query: ALTER TABLE signed_table DROP COLUMN s;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3297: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x00000000391778c2
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:15:39.339411 [ 3948 ] {ef9fb212-7c00-48ad-83bb-05370be2d2fe} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP key s column which is a part of key expression. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3297: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x00000000391778c2
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:15:39.339686 [ 7099 ] {16353634-2761-46aa-9c32-592ceb2b3da0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46954) (comment: 01018_optimize_read_in_order_with_in_subquery.sql) (in query: CREATE TABLE TESTTABLE4 (`_id` UInt64, `pt` String, `l` String) ENGINE = MergeTree PARTITION BY pt ORDER BY _id SETTINGS index_granularity = 15614, min_bytes_for_wide_part = 389124160, ratio_of_defaults_for_sparse_serialization = 0.619398832321167, replace_long_file_name_to_hash = false, max_file_name_length = 99, min_bytes_for_full_part_storage = 337711767, compact_parts_max_bytes_to_buffer = 115058898, compact_parts_max_granules_to_buffer = 248, compact_parts_merge_max_bytes_to_prefetch_part = 11967681, merge_max_block_size = 107, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 176811918, vertical_merge_algorithm_min_rows_to_activate = 326476, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 1, index_granularity_bytes = 7438491, concurrent_part_removal_threshold = 32, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = false, marks_compress_block_size = 15828, primary_key_compress_block_size = 29499), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::loadDataParts(bool, std::optional<std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>>) @ 0x000000003910bb4b
8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04668
9. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
12. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:15:39.342107 [ 7099 ] {16353634-2761-46aa-9c32-592ceb2b3da0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::loadDataParts(bool, std::optional<std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>>) @ 0x000000003910bb4b
8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04668
9. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
12. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:15:39.700399 [ 3948 ] {d83d02ca-7933-4046-a780-552de052e225} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER RENAME key s column which is a part of key expression. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46948) (comment: 01716_drop_rename_sign_column.sql) (in query: ALTER TABLE signed_table RENAME COLUMN s TO s1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3277: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000039177b62
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:15:39.701527 [ 3948 ] {d83d02ca-7933-4046-a780-552de052e225} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER RENAME key s column which is a part of key expression. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3277: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000039177b62
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:15:40.894909 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/xgj/wuxirzlmymxmpcvnltfzvyujafyym, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f2157a82-3ed3-4d33-8080-e2a2218080eb, upload has finished false.
2024.11.04 23:15:40.895024 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/xgj/wuxirzlmymxmpcvnltfzvyujafyym, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f2157a82-3ed3-4d33-8080-e2a2218080eb, upload has finished false
2024.11.04 23:15:40.907170 [ 3398 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:15:40.907243 [ 3347 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:15:40.907339 [ 3377 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:15:40.907411 [ 3345 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:15:40.907480 [ 3380 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:15:40.907531 [ 3382 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:15:40.936597 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/xgj/wuxirzlmymxmpcvnltfzvyujafyym, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f2157a82-3ed3-4d33-8080-e2a2218080eb, upload has finished false
2024.11.04 23:15:41.325889 [ 7099 ] {5d6e8532-6aed-4614-8ec4-89e8db29e63f} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows to GROUP BY exceeded: has 11 rows, maximum: 10: While executing AggregatingTransform. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46964) (comment: 01134_max_rows_to_group_by.sql) (in query: SELECT 'test1', number FROM system.numbers GROUP BY number;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<unsigned long&, unsigned long const&>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<unsigned long const&>::type>, unsigned long&, unsigned long const&) @ 0x000000000ad6782c
3. ./build_docker/./src/Interpreters/Aggregator.cpp:1842: DB::Aggregator::checkLimits(unsigned long, bool&) const @ 0x0000000032d32196
4. ./build_docker/./src/Interpreters/Aggregator.cpp:0: DB::Aggregator::executeOnBlock(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>, unsigned long, unsigned long, DB::AggregatedDataVariants&, std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*>>&, std::vector<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*>>, std::allocator<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*>>>>&, bool&) const @ 0x0000000032d523c9
5. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:692: DB::AggregatingTransform::consume(DB::Chunk) @ 0x000000003b4fbf01
6. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:0: DB::AggregatingTransform::work() @ 0x000000003b4f07c8
7. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
9. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
10. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
11. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:15:41.326048 [ 3337 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.326043 [ 3385 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/1_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.326069 [ 3358 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.326155 [ 3336 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.326234 [ 3371 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.326262 [ 3383 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.326285 [ 3394 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.326501 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
17. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:142: DB::SerializationString::serializeBinaryBulk(DB::IColumn const&, DB::WriteBuffer&, unsigned long, unsigned long) const @ 0x0000000031ce7e63
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031ba0ede
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeSingleGranule(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&, DB::ISerialization::SerializeBinaryBulkSettings&, DB::Granule const&) @ 0x00000000393f9a92
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f6050
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:41.327752 [ 7099 ] {5d6e8532-6aed-4614-8ec4-89e8db29e63f} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows to GROUP BY exceeded: has 11 rows, maximum: 10: While executing AggregatingTransform. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<unsigned long&, unsigned long const&>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<unsigned long const&>::type>, unsigned long&, unsigned long const&) @ 0x000000000ad6782c
3. ./build_docker/./src/Interpreters/Aggregator.cpp:1842: DB::Aggregator::checkLimits(unsigned long, bool&) const @ 0x0000000032d32196
4. ./build_docker/./src/Interpreters/Aggregator.cpp:0: DB::Aggregator::executeOnBlock(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>, unsigned long, unsigned long, DB::AggregatedDataVariants&, std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*>>&, std::vector<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*>>, std::allocator<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*>>>>&, bool&) const @ 0x0000000032d523c9
5. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:692: DB::AggregatingTransform::consume(DB::Chunk) @ 0x000000003b4fbf01
6. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:0: DB::AggregatingTransform::work() @ 0x000000003b4f07c8
7. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
9. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
10. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
11. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:15:41.416812 [ 4719 ] {} <Warning> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): Node with parent zookeeper lock /clickhouse/zero_copy/zero_copy_s3/6d20a280-f581-463c-ae84-0eea31a2f0e2/20180101_0_0_0/test_jpn_jwsbeernwwddburzqbhxxpjdrzjxd for part 20180101_0_0_0 doesn't exist (part was unlocked before), refuse to remove blobs
2024.11.04 23:15:41.417167 [ 4633 ] {} <Warning> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): Node with parent zookeeper lock /clickhouse/zero_copy/zero_copy_s3/6d20a280-f581-463c-ae84-0eea31a2f0e2/20180101_2_2_0/test_gxs_dmimndkvqavoogytjiqowrnzcxsfz for part 20180101_2_2_0 doesn't exist (part was unlocked before), refuse to remove blobs
2024.11.04 23:15:41.417326 [ 4791 ] {} <Warning> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): Node with parent zookeeper lock /clickhouse/zero_copy/zero_copy_s3/6d20a280-f581-463c-ae84-0eea31a2f0e2/20180101_3_3_0/test_niv_pofilcryvprbowzxjingqctpuprre for part 20180101_3_3_0 doesn't exist (part was unlocked before), refuse to remove blobs
2024.11.04 23:15:41.417494 [ 4796 ] {} <Warning> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): Node with parent zookeeper lock /clickhouse/zero_copy/zero_copy_s3/6d20a280-f581-463c-ae84-0eea31a2f0e2/20180101_4_4_0/test_xse_ibdyokadyakmkxhycdmfdicdcgver for part 20180101_4_4_0 doesn't exist (part was unlocked before), refuse to remove blobs
2024.11.04 23:15:41.417557 [ 4578 ] {} <Warning> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): Node with parent zookeeper lock /clickhouse/zero_copy/zero_copy_s3/6d20a280-f581-463c-ae84-0eea31a2f0e2/20180101_5_5_0/test_erv_rftvbpkwponvyksoxhtnrpmlkxjmd for part 20180101_5_5_0 doesn't exist (part was unlocked before), refuse to remove blobs
2024.11.04 23:15:41.539186 [ 7099 ] {64d7e130-5344-410f-8396-363ff212348f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46964) (comment: 01134_max_rows_to_group_by.sql) (in query: SELECT 'test2', number FROM system.numbers GROUP BY number ORDER BY number;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:15:41.542467 [ 7099 ] {64d7e130-5344-410f-8396-363ff212348f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:15:43.272675 [ 3948 ] {993302e9-deb3-4a53-9c30-fd51eba768a0} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 12.00, current rows: 28.00: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46972) (comment: 02155_read_in_order_max_rows_to_read.sql) (in query: SELECT a FROM t_max_rows_to_read ORDER BY a LIMIT 20 FORMAT Null SETTINGS max_rows_to_read = 12;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x00000000318c84c2
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69d18e
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:43.274894 [ 3948 ] {993302e9-deb3-4a53-9c30-fd51eba768a0} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 12.00, current rows: 28.00: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x00000000318c84c2
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69d18e
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:43.368028 [ 3948 ] {6375bc7f-fcec-4ee1-8254-55e5c4e1b840} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 12.00, current rows: 92.00: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46972) (comment: 02155_read_in_order_max_rows_to_read.sql) (in query: SELECT a FROM t_max_rows_to_read WHERE a > 10 ORDER BY a LIMIT 5 FORMAT Null SETTINGS max_rows_to_read = 12;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x00000000318c84c2
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69d18e
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:43.370471 [ 3948 ] {6375bc7f-fcec-4ee1-8254-55e5c4e1b840} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows or bytes to read exceeded, max rows: 12.00, current rows: 92.00: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/ReadProgressCallback.cpp:109: DB::ReadProgressCallback::onProgress(unsigned long, unsigned long, std::list<DB::StorageLimits, std::allocator<DB::StorageLimits>> const&) @ 0x00000000318c84c2
5. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:62: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69d18e
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:43.408065 [ 3948 ] {9d79d129-af2e-4e58-b332-549e114e4d54} <Error> executeQuery: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 4.00, current rows: 12.00. (TOO_MANY_ROWS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46972) (comment: 02155_read_in_order_max_rows_to_read.sql) (in query: SELECT a FROM t_max_rows_to_read WHERE a = 10 OR a = 20 FORMAT Null SETTINGS max_rows_to_read = 4;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6eec04
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:15:43.409408 [ 3948 ] {9d79d129-af2e-4e58-b332-549e114e4d54} <Error> TCPHandler: Code: 158. DB::Exception: Limit for rows (controlled by 'max_rows_to_read' setting) exceeded, max rows: 4.00, current rows: 12.00. (TOO_MANY_ROWS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&, String&&, String&&) @ 0x0000000031925ce2
3. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:25: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int, int) const @ 0x0000000031925415
4. ./build_docker/./src/QueryPipeline/SizeLimits.cpp:65: DB::SizeLimits::check(unsigned long, unsigned long, char const*, int) const @ 0x000000003192586c
5. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:257: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6eec04
6. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
7. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:15:44.088031 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:44.681286 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_72_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:45.221187 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_73_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:45.786234 [ 3948 ] {22aec356-e1e7-4b66-b286-3e39280f3652} <Error> executeQuery: Code: 48. DB::Exception: Only `hash` join supports multiple ORs for keys in JOIN ON section. (NOT_IMPLEMENTED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50276) (comment: 01660_join_or_any.sql) (in query: select a2, b2 from tab2 any left join tab3 on a2 = a3 or b2 = b3 ORDER BY a2, b2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ace9
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:15:45.789131 [ 3948 ] {22aec356-e1e7-4b66-b286-3e39280f3652} <Error> TCPHandler: Code: 48. DB::Exception: Only `hash` join supports multiple ORs for keys in JOIN ON section. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ace9
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:15:45.801639 [ 3735 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:15:45.801777 [ 3735 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:15:46.025366 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_81_16}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:46.583608 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/aec/qaejgqfdqxnerymbqnyursqrvqces, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ecb39379-f51f-4256-8366-73b2a6c033bf, upload has finished false.
2024.11.04 23:15:46.583767 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/aec/qaejgqfdqxnerymbqnyursqrvqces, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ecb39379-f51f-4256-8366-73b2a6c033bf, upload has finished false
2024.11.04 23:15:46.596602 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/aec/qaejgqfdqxnerymbqnyursqrvqces, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ecb39379-f51f-4256-8366-73b2a6c033bf, upload has finished false
2024.11.04 23:15:46.600377 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 60 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:47.481884 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_73_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:49.564435 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:50.442633 [ 7099 ] {250d738a-a140-4f1d-9834-5534c7f55b97} <Error> executeQuery: Code: 457. DB::Exception: Value [1] OR 1 cannot be parsed as Array(UInt32) for query parameter 'injection' because it isn't parsed completely: only 3 of 8 bytes was parsed: [1]. (BAD_QUERY_PARAMETER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50392) (comment: 00955_complex_prepared_statements.sh) (in query: SELECT * FROM ps WHERE a = {injection:Array(UInt32)}), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, String const&, String const&, unsigned long, unsigned long, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long>::type, std::type_identity<String>::type>, String const&, String const&, String const&, unsigned long&&, unsigned long&&, String&&) @ 0x000000003653a2cd
3. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:104: DB::ReplaceQueryParameterVisitor::visitQueryParameter(std::shared_ptr<DB::IAST>&) @ 0x0000000036536e23
4. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:38: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532adb
5. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
6. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
7. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
8. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
9. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
10. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
11. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
12. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
13. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
14. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
15. ./src/Interpreters/ReplaceQueryParameterVisitor.h:23: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036594c50
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:15:50.442711 [ 3948 ] {9b9400c4-42c0-4c96-823c-b3089416e5ea} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50378) (comment: 02122_parallel_formatting_CSVWithNames.sh) (in query: select number, number + 1, concat('string: ', toString(number)) from numbers(200000) group by number with totals order by number limit 190000 format CSVWithNames), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:50.444709 [ 7099 ] {250d738a-a140-4f1d-9834-5534c7f55b97} <Error> TCPHandler: Code: 457. DB::Exception: Value [1] OR 1 cannot be parsed as Array(UInt32) for query parameter 'injection' because it isn't parsed completely: only 3 of 8 bytes was parsed: [1]. (BAD_QUERY_PARAMETER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, String const&, String const&, unsigned long, unsigned long, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String const&>::type, std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long>::type, std::type_identity<String>::type>, String const&, String const&, String const&, unsigned long&&, unsigned long&&, String&&) @ 0x000000003653a2cd
3. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:104: DB::ReplaceQueryParameterVisitor::visitQueryParameter(std::shared_ptr<DB::IAST>&) @ 0x0000000036536e23
4. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:38: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532adb
5. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
6. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
7. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
8. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
9. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
10. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
11. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
12. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
13. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:0: DB::ReplaceQueryParameterVisitor::visitChildren(std::shared_ptr<DB::IAST>&) @ 0x00000000365393df
14. ./build_docker/./src/Interpreters/ReplaceQueryParameterVisitor.cpp:48: DB::ReplaceQueryParameterVisitor::visit(std::shared_ptr<DB::IAST>&) @ 0x0000000036532ce6
15. ./src/Interpreters/ReplaceQueryParameterVisitor.h:23: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000036594c50
16. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
18. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:15:50.447014 [ 3948 ] {9b9400c4-42c0-4c96-823c-b3089416e5ea} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:15:50.473571 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_73_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:50.778317 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/oev/vstscbxwxsolmzlvivokznyauyits, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 58163449-6d30-4ad7-a043-7f989e6cf3b8, upload has finished false.
2024.11.04 23:15:50.778485 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/oev/vstscbxwxsolmzlvivokznyauyits, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 58163449-6d30-4ad7-a043-7f989e6cf3b8, upload has finished false
2024.11.04 23:15:50.785179 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/oev/vstscbxwxsolmzlvivokznyauyits, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 58163449-6d30-4ad7-a043-7f989e6cf3b8, upload has finished false
2024.11.04 23:15:50.786810 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. ./src/IO/WriteBuffer.h:65: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039408090
17. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&) @ 0x00000000393f910f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f5c43
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:50.984292 [ 3499 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:15:51.176638 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_74_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:51.702208 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_74_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:53.709510 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:55.437019 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_74_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:15:58.988030 [ 3499 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:16:00.755284 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:00.755416 [ 3948 ] {92f1fc70-7fed-4104-a962-273ea6bd6f71} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52388) (comment: 02919_ddsketch_quantile.sql) (in query: SELECT round(quantileDD(0.01, 0.5)(number), 2) FROM numbers(200);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:00.761162 [ 3948 ] {92f1fc70-7fed-4104-a962-273ea6bd6f71} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:02.769857 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:03.251093 [ 3327 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:03.251169 [ 3290 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:03.251361 [ 3320 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:03.251403 [ 3325 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:03.251511 [ 3326 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:03.251708 [ 3321 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:03.724634 [ 3948 ] {bf836b39-05b4-412d-94ab-4441408de97e} <Error> executeQuery: Code: 80. DB::Exception: CREATE INDEX without TYPE is forbidden. SET allow_create_index_without_type=1 to ignore this statements. (INCORRECT_QUERY) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52396) (comment: 02813_create_index_noop.sql) (in query: CREATE INDEX idx_tab1_0 on tab1 (col0);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/InterpreterCreateIndexQuery.cpp:0: DB::InterpreterCreateIndexQuery::execute() @ 0x000000003677a2da
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
5. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:16:03.724847 [ 3113 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:03.725053 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_74_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:03.776659 [ 3738 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:16:03.776819 [ 3738 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:16:03.892784 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/usv/qzecwmphzikgajnzamqqvmzpecxue, total size 402653184, count 402653184, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 4b124787-872e-4853-827b-59ca31ac879d, upload has finished false.
2024.11.04 23:16:03.893068 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/usv/qzecwmphzikgajnzamqqvmzpecxue, total size 402653184, count 402653184, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 4b124787-872e-4853-827b-59ca31ac879d, upload has finished false
2024.11.04 23:16:03.996597 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/usv/qzecwmphzikgajnzamqqvmzpecxue, total size 402653184, count 402653184, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 4b124787-872e-4853-827b-59ca31ac879d, upload has finished false
2024.11.04 23:16:04.772015 [ 3948 ] {bf836b39-05b4-412d-94ab-4441408de97e} <Error> TCPHandler: Code: 80. DB::Exception: CREATE INDEX without TYPE is forbidden. SET allow_create_index_without_type=1 to ignore this statements. (INCORRECT_QUERY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/InterpreterCreateIndexQuery.cpp:0: DB::InterpreterCreateIndexQuery::execute() @ 0x000000003677a2da
4. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
5. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850

2024.11.04 23:16:04.772071 [ 3144 ] {} <Error> test_13.t1 (ReplicatedMergeTreeRestartingThread): Couldn't start replication (table will be in readonly mode): Coordination error: Operation timeout, path /02946_parallel_replicas/test_13/test_tbl/replicas/r1/queue. Code: 999. Coordination::Exception: Coordination error: Operation timeout, path /02946_parallel_replicas/test_13/test_tbl/replicas/r1/queue. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:240: zkutil::ZooKeeper::getChildren(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039a2c
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::load(std::shared_ptr<zkutil::ZooKeeper>) @ 0x0000000039b159ab
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:0: DB::ReplicatedMergeTreeRestartingThread::tryStartup() @ 0x0000000039bf82ba
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:137: DB::ReplicatedMergeTreeRestartingThread::runImpl() @ 0x0000000039bf469c
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:64: DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000039bf28b6
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039c072df
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:04.772160 [ 3113 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
13. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
15. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
16. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:04.772289 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 72 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:04.800858 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_74_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:04.872478 [ 3345 ] {} <Warning> test_13.t1 (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it.
2024.11.04 23:16:05.956110 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:06.163441 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:06.567546 [ 7099 ] {1419cd16-fdca-4765-adbf-91751a5bee97} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50448) (comment: 02946_parallel_replicas_force_primary_key.sql) (in query: -- w/o parallel replicas
 SELECT k, count() FROM t1 WHERE k > 0 GROUP BY k ORDER BY k SETTINGS force_primary_key = 1, allow_experimental_parallel_reading_from_replicas = 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:06.579867 [ 7099 ] {1419cd16-fdca-4765-adbf-91751a5bee97} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:07.903128 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_74_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:08.000016 [ 7099 ] {04ca1224-488c-4f4d-9108-48f122c9ac3e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39314) (comment: 01392_column_resolve.sql) (in query: SELECT conversion.conversionId AS myConversionId, click.clickId AS myClickId, click.myValue AS myValue FROM ( SELECT conversionId, value as myValue FROM test_01392.tableConversion ) AS conversion INNER JOIN ( SELECT clickId, conversionId, value as myValue FROM test_01392.tableClick ) AS click ON click.conversionId = conversion.conversionId LEFT JOIN ( SELECT * FROM test_01392.leftjoin ) AS dummy ON (dummy.id = conversion.conversionId) ORDER BY myValue;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:16:08.015533 [ 7099 ] {04ca1224-488c-4f4d-9108-48f122c9ac3e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:16:08.533745 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:10.372784 [ 7099 ] {f144a02f-d511-4c20-bf30-2f34fa9a78e9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39344) (comment: 02245_weird_partitions_pruning.sql) (in query: INSERT INTO weird_partitions_02245(d, id) SELECT toDateTime('2023-01-31 22:30:00') AS d, number FROM numbers(1000);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:10.381188 [ 7099 ] {f144a02f-d511-4c20-bf30-2f34fa9a78e9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:10.539477 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:11.490653 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_74_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:12.133374 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/yhd/hyhgjkxpzksektqnvlymdngjgmyjw, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8c37b4ae-845b-4742-9859-18331644bc1e, upload has finished false.
2024.11.04 23:16:12.133543 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/yhd/hyhgjkxpzksektqnvlymdngjgmyjw, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8c37b4ae-845b-4742-9859-18331644bc1e, upload has finished false
2024.11.04 23:16:12.156962 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/yhd/hyhgjkxpzksektqnvlymdngjgmyjw, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8c37b4ae-845b-4742-9859-18331644bc1e, upload has finished false
2024.11.04 23:16:12.160630 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 32 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:13.995853 [ 7099 ] {5524dcfe-4a48-4659-bdab-9eaec31100ab} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39362) (comment: 02845_join_on_cond_sparse.sql) (in query: INSERT INTO t1 VALUES ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86

2024.11.04 23:16:13.998579 [ 7099 ] {5524dcfe-4a48-4659-bdab-9eaec31100ab} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86

2024.11.04 23:16:14.006176 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_75_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:14.106953 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_9_9_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:14.959289 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_75_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:15.349442 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:16.969428 [ 7099 ] {1151a68c-4c92-47e5-acda-165d4b0c524f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56514) (comment: 02099_tsv_raw_format.sh) (in query: INSERT INTO test_02099 SELECT number, toString(number), toDate(number) FROM numbers(3)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:16.974345 [ 7099 ] {1151a68c-4c92-47e5-acda-165d4b0c524f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:17.948948 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:18.104021 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:18.549837 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_75_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:20.444029 [ 4242 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:20.444814 [ 3530 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:20.444928 [ 3510 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:20.445062 [ 3531 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:20.445199 [ 3467 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:20.445341 [ 3565 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:20.445483 [ 3497 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:20.445626 [ 3517 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:20.446621 [ 3502 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:20.674920 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=47). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:20.999389 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/dmr/ymplhvwrxmwujunbcayckzimrfvun, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d8100c76-7b2d-4ec4-be3d-b98a7a90ca0e, upload has finished false.
2024.11.04 23:16:20.999563 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/dmr/ymplhvwrxmwujunbcayckzimrfvun, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d8100c76-7b2d-4ec4-be3d-b98a7a90ca0e, upload has finished false
2024.11.04 23:16:21.030896 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/dmr/ymplhvwrxmwujunbcayckzimrfvun, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d8100c76-7b2d-4ec4-be3d-b98a7a90ca0e, upload has finished false
2024.11.04 23:16:21.034892 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 14 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:21.190936 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:21.993212 [ 3603 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:16:23.578469 [ 3949 ] {86747c91-fbc6-4a15-a765-dbfb911d5217} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56622) (comment: 02099_tsv_raw_format.sh) (in query: INSERT INTO test_02099 SELECT number, toString(number), toDate(number) FROM numbers(3)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:23.583919 [ 3949 ] {86747c91-fbc6-4a15-a765-dbfb911d5217} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a663fb1
7. ./build_docker/./src/Server/TCPHandler.cpp:567: DB::TCPHandler::runImpl() @ 0x000000003a4d71e7
8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:23.793818 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:23.913039 [ 4765 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:23.913731 [ 3140 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:23.913845 [ 3117 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:23.913993 [ 3139 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:23.914147 [ 3121 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:23.914385 [ 3127 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:23.914521 [ 3147 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:23.914652 [ 3142 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:16:23.915168 [ 3152 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:23.915179 [ 3120 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/async_blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:23.915525 [ 3115 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:24.990464 [ 3121 ] {} <Error> ZooKeeperClient: Failed to connect to ZooKeeper: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=967). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c14360e
7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:296: zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x000000003c031b77
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c037a9f
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:456: zkutil::ZooKeeper::startNewSession() const @ 0x000000003c06279c
10. ./build_docker/./src/Interpreters/Context.cpp:0: DB::Context::getZooKeeper() const @ 0x00000000329347c5
11. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x0000000038046fb1
12. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:134: DB::ReplicatedMergeTreeRestartingThread::runImpl() @ 0x0000000039bf4658
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:64: DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000039bf28b6
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039c072df
15. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
17. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
18. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:24.996132 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_100_153_10}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:25.007503 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:25.067418 [ 3121 ] {} <Error> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): Failed to restart the table. Will try again: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=967). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c14360e
7. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:296: zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x000000003c031b77
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x000000003c037a9f
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:456: zkutil::ZooKeeper::startNewSession() const @ 0x000000003c06279c
10. ./build_docker/./src/Interpreters/Context.cpp:0: DB::Context::getZooKeeper() const @ 0x00000000329347c5
11. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x0000000038046fb1
12. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:134: DB::ReplicatedMergeTreeRestartingThread::runImpl() @ 0x0000000039bf4658
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp:64: DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000039bf28b6
14. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039c072df
15. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
17. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
18. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:25.098398 [ 3213 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it.
2024.11.04 23:16:25.883914 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/uyd/gwpkpjosayrkdncgvqxpbqnehvnvg, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e13c5ada-1c67-426f-af32-8bf225d893f8, upload has finished false.
2024.11.04 23:16:25.884113 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/uyd/gwpkpjosayrkdncgvqxpbqnehvnvg, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e13c5ada-1c67-426f-af32-8bf225d893f8, upload has finished false
2024.11.04 23:16:25.893117 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/uyd/gwpkpjosayrkdncgvqxpbqnehvnvg, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e13c5ada-1c67-426f-af32-8bf225d893f8, upload has finished false
2024.11.04 23:16:25.896447 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 44 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:28.206606 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:28.206691 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:28.215590 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:29.998241 [ 3603 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:16:30.013935 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:32.664686 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:33.194142 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:34.336364 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/nkc/xewllmpobsvudckvuyunythngnoic, total size 268435456, count 268435456, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 33fe7939-4285-4178-a688-64622e0f4b72, upload has finished false.
2024.11.04 23:16:34.336514 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/nkc/xewllmpobsvudckvuyunythngnoic, total size 268435456, count 268435456, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 33fe7939-4285-4178-a688-64622e0f4b72, upload has finished false
2024.11.04 23:16:34.373926 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/nkc/xewllmpobsvudckvuyunythngnoic, total size 268435456, count 268435456, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 33fe7939-4285-4178-a688-64622e0f4b72, upload has finished false
2024.11.04 23:16:34.831319 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:34.831477 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 89 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:37.168037 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:37.588583 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:40.751453 [ 7099 ] {a1dc114a-ab33-44bd-a380-bb2bc5f3799a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34098) (comment: 02099_tsv_raw_format.sh) (in query: SELECT * FROM test_nullable_02099), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:40.751473 [ 2935 ] {f5e741c6-e014-4f95-a888-cd5c2b2b9945} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34056) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: CREATE WINDOW VIEW wv TO dst WATERMARK=STRICTLY_ASCENDING AS SELECT count(a) AS count, tumbleEnd(wid) as w_end FROM mt GROUP BY tumble(timestamp, INTERVAL '5' SECOND, 'US/Samoa') AS wid;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::loadDataParts(bool, std::optional<std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>>) @ 0x000000003910bb4b
8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04668
9. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
12. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Storages/WindowView/StorageWindowView.cpp:0: DB::StorageWindowView::StorageWindowView(DB::StorageID const&, std::shared_ptr<DB::Context const>, DB::ASTCreateQuery const&, DB::ColumnsDescription const&, DB::LoadingStrictnessLevel) @ 0x0000000039e66fa6
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::StorageWindowView> std::allocate_shared[abi:v15000]<DB::StorageWindowView, std::allocator<DB::StorageWindowView>, DB::StorageID const&, std::shared_ptr<DB::Context>, DB::ASTCreateQuery const&, DB::ColumnsDescription const&, DB::LoadingStrictnessLevel const&, void>(std::allocator<DB::StorageWindowView> const&, DB::StorageID const&, std::shared_ptr<DB::Context>&&, DB::ASTCreateQuery const&, DB::ColumnsDescription const&, DB::LoadingStrictnessLevel const&) @ 0x0000000039ea7872
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: std::shared_ptr<DB::IStorage> std::__function::__policy_invoker<std::shared_ptr<DB::IStorage> (DB::StorageFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerStorageWindowView(DB::StorageFactory&)::$_0, std::shared_ptr<DB::IStorage> (DB::StorageFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::StorageFactory::Arguments const&) @ 0x0000000039e8a942
19. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
20. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
21. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
22. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
30. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
31. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32

2024.11.04 23:16:40.753926 [ 2935 ] {f5e741c6-e014-4f95-a888-cd5c2b2b9945} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::loadDataParts(bool, std::optional<std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>>) @ 0x000000003910bb4b
8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>) @ 0x0000000039d04668
9. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:65: std::shared_ptr<DB::StorageMergeTree> std::allocate_shared[abi:v15000]<DB::StorageMergeTree, std::allocator<DB::StorageMergeTree>, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, void>(std::allocator<DB::StorageMergeTree> const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr<DB::Context>&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>&&) @ 0x0000000039d02587
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000039cf2f90
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003038fd3a
12. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
15. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
16. ./build_docker/./src/Storages/WindowView/StorageWindowView.cpp:0: DB::StorageWindowView::StorageWindowView(DB::StorageID const&, std::shared_ptr<DB::Context const>, DB::ASTCreateQuery const&, DB::ColumnsDescription const&, DB::LoadingStrictnessLevel) @ 0x0000000039e66fa6
17. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::StorageWindowView> std::allocate_shared[abi:v15000]<DB::StorageWindowView, std::allocator<DB::StorageWindowView>, DB::StorageID const&, std::shared_ptr<DB::Context>, DB::ASTCreateQuery const&, DB::ColumnsDescription const&, DB::LoadingStrictnessLevel const&, void>(std::allocator<DB::StorageWindowView> const&, DB::StorageID const&, std::shared_ptr<DB::Context>&&, DB::ASTCreateQuery const&, DB::ColumnsDescription const&, DB::LoadingStrictnessLevel const&) @ 0x0000000039ea7872
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:0: std::shared_ptr<DB::IStorage> std::__function::__policy_invoker<std::shared_ptr<DB::IStorage> (DB::StorageFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerStorageWindowView(DB::StorageFactory&)::$_0, std::shared_ptr<DB::IStorage> (DB::StorageFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::StorageFactory::Arguments const&) @ 0x0000000039e8a942
19. ./build_docker/./src/Storages/StorageFactory.cpp:224: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x0000000037c972f6
20. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003465e6a1
21. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
22. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
24. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
26. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
30. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
31. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32

2024.11.04 23:16:40.754495 [ 7099 ] {a1dc114a-ab33-44bd-a380-bb2bc5f3799a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:40.766146 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_77_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:41.962193 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/iaw/yotgugjgzxtdqyjfhnokglyogxekj, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id da311227-76dc-4ba4-a415-2b5e13506d72, upload has finished false.
2024.11.04 23:16:41.962345 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/iaw/yotgugjgzxtdqyjfhnokglyogxekj, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id da311227-76dc-4ba4-a415-2b5e13506d72, upload has finished false
2024.11.04 23:16:41.987649 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/iaw/yotgugjgzxtdqyjfhnokglyogxekj, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id da311227-76dc-4ba4-a415-2b5e13506d72, upload has finished false
2024.11.04 23:16:41.991931 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 66 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:42.472748 [ 3948 ] {13b5d68a-559b-426f-8e5c-3036cbb2276a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34140) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:42.476032 [ 3948 ] {13b5d68a-559b-426f-8e5c-3036cbb2276a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:43.675377 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_77_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:45.774001 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_77_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:46.872969 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:47.022932 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_78_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:47.029213 [ 2935 ] {a24ac27e-24d7-4b7a-9bc9-5d6650a43fd2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55354) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:47.033044 [ 2935 ] {a24ac27e-24d7-4b7a-9bc9-5d6650a43fd2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:16:47.388165 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/pqv/hqmzpuyjmgkcavifhnvtsilbrmwev, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5e84f944-2327-43b5-a88e-9176be44df85, upload has finished false.
2024.11.04 23:16:47.388441 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/pqv/hqmzpuyjmgkcavifhnvtsilbrmwev, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5e84f944-2327-43b5-a88e-9176be44df85, upload has finished false
2024.11.04 23:16:47.429932 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/pqv/hqmzpuyjmgkcavifhnvtsilbrmwev, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5e84f944-2327-43b5-a88e-9176be44df85, upload has finished false
2024.11.04 23:16:47.435419 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 61 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:47.535882 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_78_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:49.326976 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_82_17}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=30). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:50.055022 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_82_17}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=3): (while reading column event_time_microseconds): (while reading from part /var/lib/clickhouse/disks/s3/store/f42/f421370c-92f1-4a0e-839e-bdd872778e02/202411_1_79_16/ in table system.trace_log (f421370c-92f1-4a0e-839e-bdd872778e02) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:51.358660 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_78_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:51.861163 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:16:51.861287 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:16:52.080459 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_82_17}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:52.130284 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/bzy/vjtypktcxorqahsxerizqnqfaucxx, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6e287507-ef8e-450e-9d47-8cbe128d7243, upload has finished false.
2024.11.04 23:16:52.130449 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/bzy/vjtypktcxorqahsxerizqnqfaucxx, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6e287507-ef8e-450e-9d47-8cbe128d7243, upload has finished false
2024.11.04 23:16:52.138258 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/bzy/vjtypktcxorqahsxerizqnqfaucxx, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6e287507-ef8e-450e-9d47-8cbe128d7243, upload has finished false
2024.11.04 23:16:52.141675 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 18 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:53.008284 [ 3594 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:16:53.717701 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_78_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:53.985701 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_79_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:54.119920 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_82_17}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:56.934627 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_79_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:57.207140 [ 2935 ] {6d5c93a1-b0c8-4d79-8e9e-81e5ace9c9c1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44754) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:57.210286 [ 2935 ] {6d5c93a1-b0c8-4d79-8e9e-81e5ace9c9c1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:57.220391 [ 3736 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:16:57.220498 [ 3736 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:16:57.365573 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/alr/txfpdsjrobtxsptrrwzadsiytojpr, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 88b3c385-2347-49bc-84a4-5e32d87c0ce2, upload has finished false.
2024.11.04 23:16:57.365818 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/alr/txfpdsjrobtxsptrrwzadsiytojpr, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 88b3c385-2347-49bc-84a4-5e32d87c0ce2, upload has finished false
2024.11.04 23:16:57.373906 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/alr/txfpdsjrobtxsptrrwzadsiytojpr, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 88b3c385-2347-49bc-84a4-5e32d87c0ce2, upload has finished false
2024.11.04 23:16:57.434850 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 13 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:59.682710 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:59.682859 [ 2935 ] {31700391-8395-41a6-b0f5-c8380b5ea347} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44756) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:16:59.684057 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:59.685928 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_79_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:16:59.686569 [ 2935 ] {31700391-8395-41a6-b0f5-c8380b5ea347} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:01.016113 [ 3594 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:17:01.756925 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 30 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:03.026172 [ 7099 ] {31e186b5-1d51-42aa-bb7c-fed00c09b2e4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44776) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:03.029216 [ 7099 ] {31e186b5-1d51-42aa-bb7c-fed00c09b2e4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:03.719981 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_79_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:03.794863 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:04.107058 [ 3692 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded hard memory limit (14.54 GiB)
2024.11.04 23:17:05.422723 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:05.422791 [ 7099 ] {3162c012-c444-46e0-bf55-67486830015e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39850) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:05.426598 [ 7099 ] {3162c012-c444-46e0-bf55-67486830015e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:06.717546 [ 2935 ] {27cf1b6c-8dd0-4d30-b3ca-7161dd855be7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39866) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:06.720948 [ 2935 ] {27cf1b6c-8dd0-4d30-b3ca-7161dd855be7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:06.722146 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_75_17}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:07.719593 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/hlk/jockfqbkcyajfknhcfieukhxylvyr, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 9c8f2e2f-b34b-41c8-a036-d85e616f2e50, upload has finished false.
2024.11.04 23:17:07.719759 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/hlk/jockfqbkcyajfknhcfieukhxylvyr, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 9c8f2e2f-b34b-41c8-a036-d85e616f2e50, upload has finished false
2024.11.04 23:17:07.748709 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/hlk/jockfqbkcyajfknhcfieukhxylvyr, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 9c8f2e2f-b34b-41c8-a036-d85e616f2e50, upload has finished false
2024.11.04 23:17:07.753740 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 78 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:07.781058 [ 2935 ] {2c3938e1-3a2b-4758-94e0-16b4b23210c8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39880) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:07.784318 [ 2935 ] {2c3938e1-3a2b-4758-94e0-16b4b23210c8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:08.865248 [ 2935 ] {f506a33d-75c7-4932-84e9-f1aeb3ca9469} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39894) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:08.869815 [ 2935 ] {f506a33d-75c7-4932-84e9-f1aeb3ca9469} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:11.454421 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_80_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:11.954520 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_81_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:13.879200 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_81_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:15.430465 [ 2935 ] {e8946cb3-5cce-4a17-a224-814196414100} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38860) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:15.433761 [ 2935 ] {e8946cb3-5cce-4a17-a224-814196414100} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:15.567862 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/her/ioujxqlwzeuiohacuixldfvwtvtjc, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5e35a0b8-fc81-4a65-8b5c-e94cb3e9f436, upload has finished false.
2024.11.04 23:17:15.568056 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/her/ioujxqlwzeuiohacuixldfvwtvtjc, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5e35a0b8-fc81-4a65-8b5c-e94cb3e9f436, upload has finished false
2024.11.04 23:17:15.615872 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/her/ioujxqlwzeuiohacuixldfvwtvtjc, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5e35a0b8-fc81-4a65-8b5c-e94cb3e9f436, upload has finished false
2024.11.04 23:17:15.620642 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 68 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:16.104004 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_81_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:16.702586 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_81_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:16.771174 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:17.633572 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:17.668567 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:17.790489 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_81_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:19.836552 [ 2935 ] {0bcb97ff-d68a-47bc-99a2-aea2fab4f628} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38890) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:19.839561 [ 2935 ] {0bcb97ff-d68a-47bc-99a2-aea2fab4f628} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:22.564926 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:23.129148 [ 2935 ] {0fdd6884-d018-42d7-a046-f5b774b892a5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38920) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:23.132652 [ 2935 ] {0fdd6884-d018-42d7-a046-f5b774b892a5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:24.026667 [ 3198 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:17:24.864140 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:24.885496 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_81_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=33). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.226898 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.578829 [ 3464 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:25.578911 [ 3456 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:25.579043 [ 3445 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:25.579194 [ 3488 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:25.579270 [ 3468 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:25.579476 [ 3454 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:25.579648 [ 3418 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:25.580058 [ 3478 ] {} <Error> test_13.test_03217_system_tables_replica_1 (67fa74c1-2af1-48be-ab72-f06fdbf08897): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.580081 [ 3475 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.580518 [ 3435 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.580655 [ 3448 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.580762 [ 3492 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.581025 [ 3458 ] {} <Error> test_13.test_03217_system_tables_replica_1 (67fa74c1-2af1-48be-ab72-f06fdbf08897): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:25.581166 [ 3457 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:28.197434 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_82_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:30.279867 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:30.279930 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:30.279978 [ 2935 ] {e0204599-2718-486e-b0fb-1fce809976e2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55092) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:30.283538 [ 2935 ] {e0204599-2718-486e-b0fb-1fce809976e2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:30.490042 [ 3948 ] {71bede97-7e13-48a6-ba01-0fb2a363693a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55106) (comment: 00955_test_final_mark.sql) (in query: SELECT sum(marks) FROM system.parts WHERE table = 'mt_with_pk' AND database = currentDatabase() AND active=1 AND database = currentDatabase();), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:30.493208 [ 3948 ] {71bede97-7e13-48a6-ba01-0fb2a363693a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:31.058615 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/eto/kwrwhwbiqhohcygcmudtvwqhgsrrq, total size 436207616, count 436207616, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ed993fbb-948e-494c-ab3a-556824c200dc, upload has finished false.
2024.11.04 23:17:31.058795 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/eto/kwrwhwbiqhohcygcmudtvwqhgsrrq, total size 436207616, count 436207616, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ed993fbb-948e-494c-ab3a-556824c200dc, upload has finished false
2024.11.04 23:17:31.122607 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/eto/kwrwhwbiqhohcygcmudtvwqhgsrrq, total size 436207616, count 436207616, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ed993fbb-948e-494c-ab3a-556824c200dc, upload has finished false
2024.11.04 23:17:31.956028 [ 2935 ] {57d7cbda-e6dd-4f8a-b46d-4a1e6fff5e12} <Error> executeQuery: Code: 47. DB::Exception: Unknown expression identifier 'ns' in scope SELECT count(), sum(ns) FROM (SELECT intDiv(number, NULL) AS k FROM system.numbers_mt GROUP BY k) ARRAY JOIN ns. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55122) (comment: 01538_fuzz_aggregate.sql) (in query: SELECT count(), sum(ns) FROM ( SELECT intDiv(number, NULL) AS k FROM system.numbers_mt GROUP BY k ) ARRAY JOIN ns;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String&, String const&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String const&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String const&, String&&, String&&) @ 0x0000000034910cc2
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6420: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347c14e0
4. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5456
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
7. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:17:31.956127 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_8_8_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 35 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:31.956389 [ 3948 ] {7d4ef117-579b-4b24-8f17-20d8cc5e59dd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55130) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:31.957489 [ 2935 ] {57d7cbda-e6dd-4f8a-b46d-4a1e6fff5e12} <Error> TCPHandler: Code: 47. DB::Exception: Unknown expression identifier 'ns' in scope SELECT count(), sum(ns) FROM (SELECT intDiv(number, NULL) AS k FROM system.numbers_mt GROUP BY k) ARRAY JOIN ns. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String&, String const&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String const&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String const&, String&&, String&&) @ 0x0000000034910cc2
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6420: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347c14e0
4. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5456
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
7. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:17:31.960100 [ 3948 ] {7d4ef117-579b-4b24-8f17-20d8cc5e59dd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:32.030854 [ 3198 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:17:33.079593 [ 3948 ] {9bbe0038-4585-4e99-b11c-fafd654ab9c3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55156) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:33.082651 [ 3948 ] {9bbe0038-4585-4e99-b11c-fafd654ab9c3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:33.231402 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_83_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:35.338966 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:35.339307 [ 3948 ] {0aff0467-5500-4395-8a45-0aa51043efec} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59710) (comment: 02312_is_not_null_prewhere.sql) (in query: CREATE TABLE bug_36995 (`time` DateTime, `product` String) ENGINE = MergeTree ORDER BY time SETTINGS index_granularity = 60376, min_bytes_for_wide_part = 570914425, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = false, max_file_name_length = 84, min_bytes_for_full_part_storage = 285173357, compact_parts_max_bytes_to_buffer = 272036977, compact_parts_max_granules_to_buffer = 1, compact_parts_merge_max_bytes_to_prefetch_part = 4161211, merge_max_block_size = 13665, old_parts_lifetime = 298., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 7896338013, index_granularity_bytes = 4589188, concurrent_part_removal_threshold = 59, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = true, marks_compress_block_size = 81796, primary_key_compress_block_size = 81838 AS SELECT '2022-01-01 00:00:00', '1'), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:35.348050 [ 3948 ] {0aff0467-5500-4395-8a45-0aa51043efec} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:35.359838 [ 2935 ] {6bc7b244-8b27-468b-81b6-6d85ec8b50fb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59716) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:35.362928 [ 2935 ] {6bc7b244-8b27-468b-81b6-6d85ec8b50fb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:36.123674 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_83_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:36.426805 [ 3948 ] {2c7b7a56-71c0-4f94-bbbf-3655d5ad888f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59742) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:36.430148 [ 3948 ] {2c7b7a56-71c0-4f94-bbbf-3655d5ad888f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:36.509438 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ria/peryhexbhrmeixafgpweuakfpeags, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id aa3226d5-c2a9-4d45-b572-55b114c59264, upload has finished false.
2024.11.04 23:17:36.509589 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ria/peryhexbhrmeixafgpweuakfpeags, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id aa3226d5-c2a9-4d45-b572-55b114c59264, upload has finished false
2024.11.04 23:17:36.519789 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ria/peryhexbhrmeixafgpweuakfpeags, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id aa3226d5-c2a9-4d45-b572-55b114c59264, upload has finished false
2024.11.04 23:17:38.103640 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:38.103670 [ 2935 ] {3e470169-09f7-4565-8ec3-cbeff72c8375} <Error> executeQuery: Code: 47. DB::Exception: JOIN  test_13.dist_table AS d GLOBAL ALL LEFT JOIN numbers(100) AS t USING (id) using identifier 'id' cannot be resolved from right table expression. In scope SELECT uniq(d.val) FROM dist_table AS d GLOBAL LEFT JOIN numbers(100) AS t USING (id). (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59726) (comment: 01474_bad_global_join.sql) (in query: SELECT uniq(d.val) FROM dist_table AS d GLOBAL LEFT JOIN numbers(100) AS t USING id;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String const&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String const&>::type, std::type_identity<String>::type>, String&&, String const&, String&&) @ 0x00000000348e9f5e
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7864: DB::(anonymous namespace)::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x000000003487b829
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5b6d
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
7. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:17:38.103698 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 39 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:40.319713 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:40.319786 [ 2935 ] {3e470169-09f7-4565-8ec3-cbeff72c8375} <Error> TCPHandler: Code: 47. DB::Exception: JOIN  test_13.dist_table AS d GLOBAL ALL LEFT JOIN numbers(100) AS t USING (id) using identifier 'id' cannot be resolved from right table expression. In scope SELECT uniq(d.val) FROM dist_table AS d GLOBAL LEFT JOIN numbers(100) AS t USING (id). (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String, String const&, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String const&>::type, std::type_identity<String>::type>, String&&, String const&, String&&) @ 0x00000000348e9f5e
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7864: DB::(anonymous namespace)::QueryAnalyzer::resolveJoin(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x000000003487b829
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:7949: DB::(anonymous namespace)::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::QueryExpressionsAliasVisitor&) @ 0x00000000347d5b6d
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798e58
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
7. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
8. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:17:40.319853 [ 3948 ] {61b13311-bedb-4e98-8f27-3d2580690189} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59752) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:40.319991 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:40.323490 [ 3948 ] {61b13311-bedb-4e98-8f27-3d2580690189} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:40.324166 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_81_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:40.498887 [ 2935 ] {0efdfb2b-43e5-4aa1-bdba-6705d1ab9172} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59726) (comment: 01474_bad_global_join.sql) (in query: SELECT uniq(d.val) FROM dist_table AS d GLOBAL LEFT JOIN local_table AS t USING id;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:40.505137 [ 2935 ] {0efdfb2b-43e5-4aa1-bdba-6705d1ab9172} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:41.494359 [ 3948 ] {59e202d9-e07d-48a8-9db7-da68053c7fde} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59758) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:41.497950 [ 3948 ] {59e202d9-e07d-48a8-9db7-da68053c7fde} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:42.575211 [ 3948 ] {fd8ae770-bb37-4fc9-89b5-ed05cf60ce69} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59772) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:42.578917 [ 3948 ] {fd8ae770-bb37-4fc9-89b5-ed05cf60ce69} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:43.755269 [ 3948 ] {0b05c600-92cd-4097-81ef-a7dfccc14a99} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53568) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:43.758337 [ 3948 ] {0b05c600-92cd-4097-81ef-a7dfccc14a99} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:43.868085 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/pda/xbgvlktlvqxbkmgttsfpgdqsnnvgu, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a61bc04c-c154-446e-85ea-6ecf232f62a4, upload has finished false.
2024.11.04 23:17:43.868377 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/pda/xbgvlktlvqxbkmgttsfpgdqsnnvgu, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a61bc04c-c154-446e-85ea-6ecf232f62a4, upload has finished false
2024.11.04 23:17:43.904090 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/pda/xbgvlktlvqxbkmgttsfpgdqsnnvgu, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a61bc04c-c154-446e-85ea-6ecf232f62a4, upload has finished false
2024.11.04 23:17:43.974235 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 19 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:44.162456 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_84_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:44.849409 [ 3948 ] {abd08fb7-0618-427a-a346-d0bd8aaf6084} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53584) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:44.853730 [ 3948 ] {abd08fb7-0618-427a-a346-d0bd8aaf6084} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:45.229690 [ 2935 ] {26536215-a23a-4880-9f73-5f2aeafe6d3f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59756) (comment: 02021_map_bloom_filter_index.sql) (in query: SELECT * FROM map_test_index_map_values WHERE map['K2'] = 'V2' SETTINGS force_data_skipping_indices='map_bloom_filter_values';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:17:45.232622 [ 2935 ] {26536215-a23a-4880-9f73-5f2aeafe6d3f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:17:45.889064 [ 3948 ] {d7a4f977-5a0f-4b85-8703-6adbb8b0ccbf} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53596) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:45.893509 [ 3948 ] {d7a4f977-5a0f-4b85-8703-6adbb8b0ccbf} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:46.750014 [ 8002 ] {b66c9824-c5d9-4799-9a7d-40a860639259} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::ffff:127.0.0.1]:39412) (comment: 02009_body_query_params.sh) (in query: SELECT 1 AS c WHERE c = _CAST(1, 'UInt8')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::finalizeImpl() @ 0x000000003abfdfd1
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c1ddb
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850

2024.11.04 23:17:46.755139 [ 8002 ] {b66c9824-c5d9-4799-9a7d-40a860639259} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3): While executing ParallelFormattingOutputFormat. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003ac020c7
7. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::addChunk(DB::Chunk, DB::ParallelFormattingOutputFormat::ProcessingUnitType, bool) @ 0x000000003abff211
8. ./build_docker/./src/Processors/Formats/Impl/ParallelFormattingOutputFormat.cpp:0: DB::ParallelFormattingOutputFormat::finalizeImpl() @ 0x000000003abfdfd1
9. ./build_docker/./src/Processors/Formats/IOutputFormat.cpp:0: DB::IOutputFormat::work() @ 0x000000003a6c1ddb
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
14. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
16. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
17. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
18. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
22. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
23. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
24. ? @ 0x00007fe94d0adac3
25. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:46.955464 [ 3948 ] {65f5c52e-5f2c-47d4-a889-5653a819cf0c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53612) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:46.958618 [ 3948 ] {65f5c52e-5f2c-47d4-a889-5653a819cf0c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:49.203945 [ 3949 ] {c0aa690a-7999-435f-acf6-a04459fc10e9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53646) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:49.207375 [ 3949 ] {c0aa690a-7999-435f-acf6-a04459fc10e9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:49.221169 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/kht/gwpzuippqdvauazawkyinzrswplsg, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3de979d3-539f-4454-b6aa-8416c0d41025, upload has finished false.
2024.11.04 23:17:49.221357 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/kht/gwpzuippqdvauazawkyinzrswplsg, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3de979d3-539f-4454-b6aa-8416c0d41025, upload has finished false
2024.11.04 23:17:49.236495 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/kht/gwpzuippqdvauazawkyinzrswplsg, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3de979d3-539f-4454-b6aa-8416c0d41025, upload has finished false
2024.11.04 23:17:49.238311 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
17. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:142: DB::SerializationString::serializeBinaryBulk(DB::IColumn const&, DB::WriteBuffer&, unsigned long, unsigned long) const @ 0x0000000031ce7e63
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031ba0ede
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeSingleGranule(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&, DB::ISerialization::SerializeBinaryBulkSettings&, DB::Granule const&) @ 0x00000000393f9a92
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f6050
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:49.501597 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_85_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:50.380903 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_85_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:50.869818 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_85_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:51.246993 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_85_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:52.443482 [ 3949 ] {ff318b2e-c88d-48ad-91bd-8ecb3d86149e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53672) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:52.446781 [ 3949 ] {ff318b2e-c88d-48ad-91bd-8ecb3d86149e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:53.910355 [ 3948 ] {fb6657cb-6221-4a90-a024-a009b221efbb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53628) (comment: 00312_position_case_insensitive_utf8.sql) (in query: SELECT positionCaseInsensitiveUTF8(concat('       иголка.ру', arrayStringConcat(arrayMap(x -> ' ', range(20000)))), 'иголка.ру') AS res;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:17:53.912811 [ 3948 ] {fb6657cb-6221-4a90-a024-a009b221efbb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:17:53.980643 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/vdn/esciuwgdwgecofmrihrcdcwuyofvr, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a33b8ad9-1e05-43c4-b413-0f10971a6e5f, upload has finished false.
2024.11.04 23:17:53.980863 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/vdn/esciuwgdwgecofmrihrcdcwuyofvr, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a33b8ad9-1e05-43c4-b413-0f10971a6e5f, upload has finished false
2024.11.04 23:17:54.023584 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/vdn/esciuwgdwgecofmrihrcdcwuyofvr, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a33b8ad9-1e05-43c4-b413-0f10971a6e5f, upload has finished false
2024.11.04 23:17:54.056444 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 76 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.945928 [ 3626 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:54.946082 [ 3201 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:54.946248 [ 3264 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:54.946272 [ 3250 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:54.946398 [ 3268 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:54.946484 [ 3273 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:54.946744 [ 3239 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:54.947678 [ 3617 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.947708 [ 3499 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c0478b6
9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000039b37af0
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000038207de1
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.947734 [ 3549 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Operation timeout, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.947722 [ 3233 ] {} <Error> test_13.table_rename_with_ttl (a1b03880-5d9d-456b-9945-58f50d466a70): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/test_13/test/table_rename_with_ttl_01378/replicas/1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.947717 [ 3225 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/1_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.948050 [ 3252 ] {} <Error> test_13.alter_compression_codec1 (6d20a280-f581-463c-ae84-0eea31a2f0e2): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.948032 [ 3262 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.948117 [ 3240 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.948575 [ 3578 ] {} <Error> test_13.table_rename_with_ttl (a1b03880-5d9d-456b-9945-58f50d466a70): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:54.957202 [ 3948 ] {fdf4a0ce-c630-48dc-b9c9-5c6908a125a7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45602) (comment: 02132_empty_mutation_livelock.sql) (in query: select count() from a8x;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:54.961286 [ 3948 ] {fdf4a0ce-c630-48dc-b9c9-5c6908a125a7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:55.040508 [ 3615 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:17:55.384925 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:55.820653 [ 3948 ] {49aec61f-32ab-4506-abf7-80d9f88eaf17} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45632) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:55.824039 [ 3948 ] {49aec61f-32ab-4506-abf7-80d9f88eaf17} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:56.897140 [ 3948 ] {0251cafc-b146-4b68-843e-3a708af45ecf} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45648) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:56.900619 [ 3948 ] {0251cafc-b146-4b68-843e-3a708af45ecf} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:17:57.712744 [ 4884 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:57.713611 [ 3513 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:57.713684 [ 3452 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:57.713767 [ 3429 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:57.713826 [ 3498 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:57.713935 [ 3476 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:57.714016 [ 3501 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:57.714160 [ 3505 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:17:57.714668 [ 3493 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:57.714705 [ 3527 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:57.715126 [ 3572 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:57.756559 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_1_170_16}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column message): (while reading from part /var/lib/clickhouse/disks/s3/store/590/5902c0d6-2094-4687-afb1-23195ef9ef4b/202411_1_99_15/ in table system.text_log (5902c0d6-2094-4687-afb1-23195ef9ef4b) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:57.990406 [ 3949 ] {5612ea65-1c6c-4626-b975-42061f9c6918} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45658) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:57.993321 [ 3949 ] {5612ea65-1c6c-4626-b975-42061f9c6918} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:17:59.479927 [ 8002 ] {c41f6ebc-fb73-45e5-baa4-a71bae6765de} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:43756) (comment: 02473_multistep_split_prewhere.sh) (in query: SELECT count() from tab_02473 PREWHERE b > 0 AND b <= 70 AND c > 0 AND c <= 30 AND d > 5 AND d <= 35 WHERE a == 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationSparse.cpp:167: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cdd498
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
29. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
31. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2

2024.11.04 23:17:59.480003 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:59.508488 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_85_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:59.567497 [ 8002 ] {c41f6ebc-fb73-45e5-baa4-a71bae6765de} <Error> DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationSparse.cpp:167: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cdd498
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
23. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
24. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
28. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:0: DB::CompletedPipelineExecutor::execute() @ 0x000000003a66463d
29. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365b0795
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
31. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:17:59.928475 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/agm/kholtozypmgmpnmrunaooqbysbbye, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ff4ac095-f446-4450-8b86-91717810f766, upload has finished false.
2024.11.04 23:17:59.928632 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/agm/kholtozypmgmpnmrunaooqbysbbye, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ff4ac095-f446-4450-8b86-91717810f766, upload has finished false
2024.11.04 23:17:59.948857 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/agm/kholtozypmgmpnmrunaooqbysbbye, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ff4ac095-f446-4450-8b86-91717810f766, upload has finished false
2024.11.04 23:17:59.952376 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 16 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:00.741102 [ 3949 ] {0aa08b01-2eea-4133-9a52-5cdb3f56a266} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45678) (comment: 02911_join_on_nullsafe_optimization.sql) (in query: -- { echoOn }
 SELECT * FROM t1 JOIN t2 ON (t1.x <=> t2.x OR (t1.x IS NULL AND t2.x IS NULL)) ORDER BY t1.x NULLS LAST;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:00.763341 [ 3949 ] {0aa08b01-2eea-4133-9a52-5cdb3f56a266} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:00.800491 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:01.814439 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_87_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:02.315220 [ 3949 ] {b9dfd5c1-339d-4cea-af1e-0e4de28a838a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45722) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:02.318723 [ 3949 ] {b9dfd5c1-339d-4cea-af1e-0e4de28a838a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:03.046812 [ 3615 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:18:04.365558 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_87_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:04.460699 [ 2935 ] {c8689f97-774d-42c8-a17f-da331bd3c93a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39420) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:04.463429 [ 2935 ] {c8689f97-774d-42c8-a17f-da331bd3c93a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:05.433955 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:05.539778 [ 2935 ] {c8c9bb67-7be3-4ea4-acd9-ef874c5207e5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39446) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:05.561019 [ 2935 ] {c8c9bb67-7be3-4ea4-acd9-ef874c5207e5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:06.314834 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/fxz/oobiduurqaoihyykydzfjdwxaihhn, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 22b5feeb-86c9-46da-ae4a-898da109dc1d, upload has finished false.
2024.11.04 23:18:06.315069 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/fxz/oobiduurqaoihyykydzfjdwxaihhn, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 22b5feeb-86c9-46da-ae4a-898da109dc1d, upload has finished false
2024.11.04 23:18:06.348033 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/fxz/oobiduurqaoihyykydzfjdwxaihhn, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 22b5feeb-86c9-46da-ae4a-898da109dc1d, upload has finished false
2024.11.04 23:18:06.352694 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 14 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:06.636788 [ 3949 ] {5c49e241-261b-43af-8660-de046674c6f5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39462) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:06.639542 [ 3949 ] {5c49e241-261b-43af-8660-de046674c6f5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:07.138978 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_87_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:07.642765 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_87_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:08.128173 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=32). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:08.595812 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:08.825439 [ 2935 ] {d3293044-006e-4449-ab03-03eabd82bbb2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39510) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:08.829376 [ 2935 ] {d3293044-006e-4449-ab03-03eabd82bbb2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:08.968110 [ 3698 ] {73612c6a-9036-473f-a428-a4fb367ef4df::all_1_1_0_3} <Error> MutatePlainMergeTreeTask: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:08.968849 [ 3698 ] {73612c6a-9036-473f-a428-a4fb367ef4df::all_1_1_0_3} <Error> virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:08.970091 [ 3949 ] {dcb99908-0c3b-4621-b51e-227be019c409} <Error> executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_3.txt' with part 'all_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39496) (comment: 01162_strange_mutations.sh) (in query: alter table t delete where n global in (select * from (select * from t where n global in (1::Int32)))), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:18:08.973078 [ 3949 ] {dcb99908-0c3b-4621-b51e-227be019c409} <Error> TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_3.txt' with part 'all_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:18:10.306477 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=1): (while reading column hostname): (while reading from part /var/lib/clickhouse/disks/s3/store/f42/f421370c-92f1-4a0e-839e-bdd872778e02/202411_1_87_18/ in table system.trace_log (f421370c-92f1-4a0e-839e-bdd872778e02) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:10.423336 [ 3705 ] {73612c6a-9036-473f-a428-a4fb367ef4df::all_1_1_0_4} <Error> MutatePlainMergeTreeTask: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:10.423919 [ 3705 ] {73612c6a-9036-473f-a428-a4fb367ef4df::all_1_1_0_4} <Error> virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:10.424694 [ 3949 ] {26eb044c-fcbd-4320-9c0b-1b1909482b3d} <Error> executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_4.txt' with part 'all_1_1_0_3' reason: 'Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39542) (comment: 01162_strange_mutations.sh) (in query: alter table t delete where n global in (select t1.n from t as t1 full join t as t2 on t1.n=t2.n where t1.n global in (select 2::Int32))), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:18:10.427049 [ 3949 ] {26eb044c-fcbd-4320-9c0b-1b1909482b3d} <Error> TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_4.txt' with part 'all_1_1_0_3' reason: 'Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:18:10.968033 [ 3949 ] {558b819c-dd3f-4f21-bf30-713833d71ebc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39558) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:10.971362 [ 3949 ] {558b819c-dd3f-4f21-bf30-713833d71ebc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:11.146733 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:11.166245 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_90_18}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:11.564092 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:11.762249 [ 3406 ] {} <Warning> test_13.t (73612c6a-9036-473f-a428-a4fb367ef4df): dropAllData: got exception removing parts from disk, removing successfully removed parts from memory.
2024.11.04 23:18:12.600636 [ 3406 ] {} <Error> DatabaseCatalog: Cannot drop table test_13.t (73612c6a-9036-473f-a428-a4fb367ef4df). Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::clearPartsFromFilesystemImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>> const&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>*) @ 0x00000000391430f1
8. ./contrib/llvm-project/libcxx/include/__hash_table:935: DB::MergeTreeData::clearPartsFromFilesystem(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>> const&, bool, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>*) @ 0x0000000039140a2c
9. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:2894: DB::MergeTreeData::dropAllData() @ 0x0000000039158b78
10. ./build_docker/./src/Storages/StorageMergeTree.cpp:327: DB::StorageMergeTree::drop() @ 0x0000000039d107b3
11. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x0000000032ba2482
12. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableDataTask() @ 0x0000000032ba0975
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000032bbce7f
14. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
16. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
17. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:12.618614 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:13.283910 [ 2935 ] {206f3278-3e41-4354-8c81-d1b310e73c04} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39594) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:13.287053 [ 2935 ] {206f3278-3e41-4354-8c81-d1b310e73c04} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:13.487020 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_88_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=40). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:13.596633 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:14.772787 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/aax/lycurygxpdfuvplrpffpwsdfzrbyi, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ee1be9a8-ee8e-48cf-b1d0-333284d4c48b, upload has finished false.
2024.11.04 23:18:14.772985 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/aax/lycurygxpdfuvplrpffpwsdfzrbyi, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ee1be9a8-ee8e-48cf-b1d0-333284d4c48b, upload has finished false
2024.11.04 23:18:14.812613 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/aax/lycurygxpdfuvplrpffpwsdfzrbyi, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ee1be9a8-ee8e-48cf-b1d0-333284d4c48b, upload has finished false
2024.11.04 23:18:14.820558 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 79 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:14.890955 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_92_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=43). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:686: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000394021b3
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:14.914416 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_88_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:15.207822 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_93_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=7): (while reading column thread_id): (while reading from part /var/lib/clickhouse/disks/s3/store/f42/f421370c-92f1-4a0e-839e-bdd872778e02/202411_1_87_18/ in table system.trace_log (f421370c-92f1-4a0e-839e-bdd872778e02) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:15.673656 [ 2935 ] {e849d0fd-99ba-4ca1-a949-6417f5a171f4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37930) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:15.677493 [ 2935 ] {e849d0fd-99ba-4ca1-a949-6417f5a171f4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:15.759820 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_88_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:16.121468 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 19 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:16.124833 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:16.355960 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_93_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:16.791110 [ 2935 ] {1ec84daa-6ee5-4cfb-bb04-0f62979675ae} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37942) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:16.797083 [ 2935 ] {1ec84daa-6ee5-4cfb-bb04-0f62979675ae} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:17.846433 [ 3949 ] {1d8d6cae-9a73-4cd2-9a33-b6259b21c789} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37948) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:17.849544 [ 3949 ] {1d8d6cae-9a73-4cd2-9a33-b6259b21c789} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:18.005617 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:18.942144 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 89 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:18.979512 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_88_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:19.020357 [ 2935 ] {fc03cf48-fee0-4a68-943b-d3c8273322c1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37986) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:19.023084 [ 2935 ] {fc03cf48-fee0-4a68-943b-d3c8273322c1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:19.646797 [ 3949 ] {da17cbbf-55b5-48dc-8195-b7e2c2cc014e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38000) (comment: 01162_strange_mutations.sh) (in query: select count(), sum(n), sum(test) from test), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:19.654729 [ 3949 ] {da17cbbf-55b5-48dc-8195-b7e2c2cc014e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:19.871585 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_89_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:19.998620 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_8_8_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:20.354153 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:21.260558 [ 2935 ] {894288eb-ee5f-43f2-a8de-5170d204bae2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38020) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:21.263745 [ 2935 ] {894288eb-ee5f-43f2-a8de-5170d204bae2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:21.328783 [ 3949 ] {576a0a37-2232-4e76-8afb-c7383c6c1fc8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38016) (comment: 01162_strange_mutations.sh) (in query: select count(), sum(n), sum(test) from test), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:21.336524 [ 3949 ] {576a0a37-2232-4e76-8afb-c7383c6c1fc8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:22.801534 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_89_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:24.496625 [ 2935 ] {32dcc31e-dd41-4a83-8bad-f37b1d98eb51} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55226) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:24.500036 [ 2935 ] {32dcc31e-dd41-4a83-8bad-f37b1d98eb51} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:26.015586 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_89_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:26.057200 [ 3127 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:18:26.690952 [ 3705 ] {} <Error> b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_1_1_0_2 (MutateFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:26.691574 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_1_1_0_2}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:26.696108 [ 2935 ] {9f8fe970-28ab-4aa6-9370-f4968bfdc774} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55270) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:26.699032 [ 2935 ] {9f8fe970-28ab-4aa6-9370-f4968bfdc774} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:26.745851 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_90_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:27.800698 [ 2935 ] {d4a69607-0c64-4f47-9bb9-314bee9ea06f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55282) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:27.804007 [ 2935 ] {d4a69607-0c64-4f47-9bb9-314bee9ea06f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:27.875989 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_90_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.433514 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.433895 [ 3707 ] {} <Error> b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_1_1_0_3 (MutateFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.433895 [ 3698 ] {} <Error> b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_0_0_0_3 (MutateFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.434461 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_1_1_0_3}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.434482 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_0_0_0_3}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.437136 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_90_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.484916 [ 4836 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.485623 [ 3372 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.485778 [ 3408 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.485799 [ 3282 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.485873 [ 3238 ] {} <Warning> test_13.t (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.485951 [ 3367 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.486032 [ 3332 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.486191 [ 3394 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.486246 [ 3113 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:18:29.486894 [ 3381 ] {} <Error> test_13.t3 (3753e979-4718-4336-bc93-97d5648b2eee): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /02946_parallel_replicas/test_13/test_tbl/replicas/r3). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.486894 [ 3144 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.486992 [ 3336 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.487068 [ 3603 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.487305 [ 3362 ] {} <Error> test_13.t3 (3753e979-4718-4336-bc93-97d5648b2eee): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.487340 [ 3344 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.487620 [ 3397 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:29.559390 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/flj/nnfsthbmhipgjergqgwgxfixhziqt, total size 268435456, count 268435456, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 195245fa-7a23-4204-95a2-6d9ed5435508, upload has finished false.
2024.11.04 23:18:29.559607 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/flj/nnfsthbmhipgjergqgwgxfixhziqt, total size 268435456, count 268435456, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 195245fa-7a23-4204-95a2-6d9ed5435508, upload has finished false
2024.11.04 23:18:29.594747 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/flj/nnfsthbmhipgjergqgwgxfixhziqt, total size 268435456, count 268435456, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 195245fa-7a23-4204-95a2-6d9ed5435508, upload has finished false
2024.11.04 23:18:31.114688 [ 3949 ] {8c311f4b-c1b3-48c0-8137-04be4f0bacb2} <Error> executeQuery: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55294) (comment: 01162_strange_mutations.sh) (in query: alter table t delete where n global in (select t1.n from t as t1 full join t as t2 on t1.n=t2.n where t1.n global in (select 2::Int32))), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:655: DB::StorageReplicatedMergeTree::waitMutationToFinishOnReplicas(std::vector<String, std::allocator<String>> const&, String const&) const @ 0x00000000380bb2a6
10. ./contrib/llvm-project/libcxx/include/vector:434: DB::StorageReplicatedMergeTree::waitMutation(String const&, unsigned long) const @ 0x00000000382a5d61
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x000000003831cd4c
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
13. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
14. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:18:31.114785 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
17. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:142: DB::SerializationString::serializeBinaryBulk(DB::IColumn const&, DB::WriteBuffer&, unsigned long, unsigned long) const @ 0x0000000031ce7e63
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::serializeBinaryBulkWithMultipleStreams(DB::IColumn const&, unsigned long, unsigned long, DB::ISerialization::SerializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&) const @ 0x0000000031ba0ede
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeSingleGranule(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::shared_ptr<DB::ISerialization::SerializeBinaryBulkState>&, DB::ISerialization::SerializeBinaryBulkSettings&, DB::Granule const&) @ 0x00000000393f9a92
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f6050
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:34.066687 [ 3127 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:18:35.958674 [ 3949 ] {8c311f4b-c1b3-48c0-8137-04be4f0bacb2} <Error> TCPHandler: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:655: DB::StorageReplicatedMergeTree::waitMutationToFinishOnReplicas(std::vector<String, std::allocator<String>> const&, String const&) const @ 0x00000000380bb2a6
10. ./contrib/llvm-project/libcxx/include/vector:434: DB::StorageReplicatedMergeTree::waitMutation(String const&, unsigned long) const @ 0x00000000382a5d61
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x000000003831cd4c
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
13. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
14. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:18:35.958675 [ 7099 ] {86cf4a5c-4e3b-49a2-b33d-55e238e5ab8b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55336) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:35.958684 [ 3709 ] {} <Error> void DB::IMergeTreeDataPart::removeIfNeeded(): while removing part all_0_0_0_3 with path store/b90/b9003d8a-7e5c-4f02-9ffd-3d64caa987e6/tmp_clone_all_0_0_0_3/: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1347: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c162f7f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c047341
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c0463e1
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:581: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000003c048371
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:571: zkutil::ZooKeeper::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000003c047c61
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeperWithFaultInjection.cpp:250: DB::ZooKeeperWithFaultInjection::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000003c198d5b
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::unlockSharedDataByID(String, String const&, DB::MergeTreePartInfo const&, String const&, String const&, std::shared_ptr<DB::ZooKeeperWithFaultInjection> const&, DB::MergeTreeSettings const&, std::shared_ptr<Poco::Logger>, String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>) @ 0x00000000383be807
11. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::unlockSharedData(DB::IMergeTreeDataPart const&, std::shared_ptr<DB::ZooKeeperWithFaultInjection> const&) const @ 0x00000000383b97ab
12. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::unlockSharedData(DB::IMergeTreeDataPart const&) const @ 0x00000000381c299a
13. ./build_docker/./src/Storages/MergeTree/IMergeTreeDataPart.cpp:0: DB::IMergeTreeDataPart::canRemovePart() const @ 0x0000000038ed8595
14. ./build_docker/./src/Storages/MergeTree/IMergeTreeDataPart.cpp:1927: DB::CanRemoveDescription std::__function::__policy_invoker<DB::CanRemoveDescription ()>::__call_impl<std::__function::__default_alloc_func<DB::IMergeTreeDataPart::remove()::$_0, DB::CanRemoveDescription ()>>(std::__function::__policy_storage const*) @ 0x0000000038eeea59
15. ./contrib/llvm-project/libcxx/include/optional:285: DB::DataPartStorageOnDiskBase::remove(std::function<DB::CanRemoveDescription ()>&&, DB::MergeTreeDataPartChecksums const&, std::list<DB::IDataPartStorage::ProjectionChecksums, std::allocator<DB::IDataPartStorage::ProjectionChecksums>>, bool, std::shared_ptr<Poco::Logger>) @ 0x0000000038e387c7
16. ./contrib/llvm-project/libcxx/include/list:616: DB::IMergeTreeDataPart::remove() @ 0x0000000038e99b09
17. ./build_docker/./src/Storages/MergeTree/IMergeTreeDataPart.cpp:0: DB::IMergeTreeDataPart::removeIfNeeded() @ 0x0000000038e95b3c
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:190: DB::MergeTreeDataPartWide::~MergeTreeDataPartWide() @ 0x000000003939d94c
19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:313: std::__shared_ptr_emplace<DB::MergeTreeDataPartWide, std::allocator<DB::MergeTreeDataPartWide>>::__on_zero_shared() @ 0x000000003935a4e1
20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:216: std::__assoc_state<std::shared_ptr<DB::IMergeTreeDataPart>>::__on_zero_shared() @ 0x000000003907ff9a
21. ./contrib/llvm-project/libcxx/include/future:1284: std::promise<std::shared_ptr<DB::IMergeTreeDataPart>>::~promise() @ 0x0000000039081f20
22. ./src/Storages/MergeTree/MutateTask.h:60: void std::__destroy_at[abi:v15000]<DB::MutateTask, 0>(DB::MutateTask*) @ 0x0000000039352f97
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:313: std::__shared_ptr_emplace<DB::MutateTask, std::allocator<DB::MutateTask>>::__on_zero_shared() @ 0x0000000039352e0d
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:216: DB::MutateFromLogEntryTask::~MutateFromLogEntryTask() @ 0x00000000398b8722
25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:313: std::__shared_ptr_emplace<DB::MutateFromLogEntryTask, std::allocator<DB::MutateFromLogEntryTask>>::__on_zero_shared() @ 0x0000000038498d81
26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:216: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>)::'lambda'(std::shared_ptr<DB::TaskRuntimeData>&&)::operator()(std::shared_ptr<DB::TaskRuntimeData>&&) const @ 0x00000000390afff3
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:317: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390aea1d
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:35.958773 [ 3697 ] {} <Error> b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_1_1_0_3 (MutateFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:35.959706 [ 3709 ] {} <Warning> zkutil::Lock: Lock is lost, because session was expired. Path: /clickhouse/zero_copy/zero_copy_s3/b9003d8a-7e5c-4f02-9ffd-3d64caa987e6/all_0_0_0_3/part_exclusive_lock, message: 1
2024.11.04 23:18:37.726396 [ 3757 ] {} <Error> void DB::SystemLog<DB::SessionLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::SessionLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::flushImpl(std::vector<DB::SessionLogElement, std::allocator<DB::SessionLogElement>> const&, unsigned long) @ 0x000000003604be4a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::SessionLogElement>::savingThreadFunction() @ 0x00000000360485a8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::SessionLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f4369c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43689f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:37.726565 [ 7099 ] {86cf4a5c-4e3b-49a2-b33d-55e238e5ab8b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:37.726564 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:37.726756 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b9003d8a-7e5c-4f02-9ffd-3d64caa987e6::all_1_1_0_3}: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:37.727325 [ 3697 ] {} <Warning> zkutil::Lock: Lock is lost, because session was expired. Path: /clickhouse/zero_copy/zero_copy_s3/b9003d8a-7e5c-4f02-9ffd-3d64caa987e6/all_1_1_0_3/part_exclusive_lock, message: 1
2024.11.04 23:18:37.731911 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_82_20}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:38.358362 [ 3949 ] {67fbc3a2-1032-4d4d-8613-ce92a76caadd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47492) (comment: 01162_strange_mutations.sh) (in query: select count() from t), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:38.363027 [ 3949 ] {67fbc3a2-1032-4d4d-8613-ce92a76caadd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:38.450473 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_90_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:38.667850 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 86 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:39.418394 [ 3143 ] {} <Warning> test_13.t (b9003d8a-7e5c-4f02-9ffd-3d64caa987e6): Removing temporary directory /var/lib/clickhouse/disks/s3/store/b90/b9003d8a-7e5c-4f02-9ffd-3d64caa987e6/delete_tmp_tmp_clone_all_0_0_0_3/
2024.11.04 23:18:39.418574 [ 3143 ] {} <Warning> test_13.t (b9003d8a-7e5c-4f02-9ffd-3d64caa987e6): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/b90/b9003d8a-7e5c-4f02-9ffd-3d64caa987e6/delete_tmp_tmp_clone_all_0_0_0_3/
2024.11.04 23:18:39.681565 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:41.023748 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 82 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:41.496136 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_91_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:41.654611 [ 3949 ] {e078dbe7-df2f-4ebe-9d66-50a194ceccd3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47556) (comment: 01162_strange_mutations.sh) (in query: select count(), sum(n), sum(test) from test), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:41.660679 [ 3949 ] {e078dbe7-df2f-4ebe-9d66-50a194ceccd3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:41.887651 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:41.930679 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:42.327930 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_91_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:42.924028 [ 3949 ] {b8e8f78b-e330-4e13-af1b-4f43ec2fdd4d} <Error> executeQuery: Code: 36. DB::Exception: ALTER UPDATE/ALTER DELETE statement with subquery may be nondeterministic, see allow_nondeterministic_mutations setting. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47568) (comment: 01162_strange_mutations.sh) (in query: ALTER TABLE test UPDATE test = (SELECT groupArray(id) FROM t1 GROUP BY 'dummy')[n - 99] WHERE 1), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:0: DB::MutationsInterpreter::validate() @ 0x0000000035a351cf
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d5671
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:18:42.926803 [ 3949 ] {b8e8f78b-e330-4e13-af1b-4f43ec2fdd4d} <Error> TCPHandler: Code: 36. DB::Exception: ALTER UPDATE/ALTER DELETE statement with subquery may be nondeterministic, see allow_nondeterministic_mutations setting. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:0: DB::MutationsInterpreter::validate() @ 0x0000000035a351cf
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d5671
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:18:42.927407 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_91_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:42.930720 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:43.100604 [ 3949 ] {1714fe22-50a4-4d90-b606-779bff9e0d4d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47576) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:43.103607 [ 3949 ] {1714fe22-50a4-4d90-b606-779bff9e0d4d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:44.072409 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 6 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:45.516690 [ 7099 ] {43ae2e3e-85b6-45c7-b1a9-60ca43fc3671} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41198) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:45.516931 [ 3709 ] {} <Error> 2b26f331-fce6-4e3f-bd41-633e7b761ba3::all_1_1_0_2 (MutateFromLogEntryTask): virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column n): (while reading from part /var/lib/clickhouse/disks/s3/store/2b2/2b26f331-fce6-4e3f-bd41-633e7b761ba3/all_1_1_0/ in table test_13.test (2b26f331-fce6-4e3f-bd41-633e7b761ba3) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 50): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x00000000398c127e
26. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x0000000039939ac2
27. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1729: DB::MutateSomePartColumnsTask::executeStep() @ 0x000000003993d3ed
28. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:2010: DB::MutateTask::execute() @ 0x00000000398caa43
29. ./src/Storages/MergeTree/MutateFromLogEntryTask.h:44: DB::MutateFromLogEntryTask::executeInnerTask() @ 0x00000000398b8da6
30. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a607d2
31. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:45.517607 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {2b26f331-fce6-4e3f-bd41-633e7b761ba3::all_1_1_0_2}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column n): (while reading from part /var/lib/clickhouse/disks/s3/store/2b2/2b26f331-fce6-4e3f-bd41-633e7b761ba3/all_1_1_0/ in table test_13.test (2b26f331-fce6-4e3f-bd41-633e7b761ba3) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 50): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x00000000398c127e
26. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x0000000039939ac2
27. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1729: DB::MutateSomePartColumnsTask::executeStep() @ 0x000000003993d3ed
28. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:2010: DB::MutateTask::execute() @ 0x00000000398caa43
29. ./src/Storages/MergeTree/MutateFromLogEntryTask.h:44: DB::MutateFromLogEntryTask::executeInnerTask() @ 0x00000000398b8da6
30. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeImpl() @ 0x0000000039a607d2
31. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeMutateTaskBase.cpp:0: DB::ReplicatedMergeMutateTaskBase::executeStep() @ 0x0000000039a5c972
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:45.520664 [ 7099 ] {43ae2e3e-85b6-45c7-b1a9-60ca43fc3671} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:47.428802 [ 3949 ] {6db43001-07f0-4f65-80a0-dcd570e101b7} <Error> executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation '0000000000' with part 'all_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column n): (while reading from part /var/lib/clickhouse/disks/s3/store/2b2/2b26f331-fce6-4e3f-bd41-633e7b761ba3/all_1_1_0/ in table test_13.test (2b26f331-fce6-4e3f-bd41-633e7b761ba3) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 50): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41186) (comment: 01162_strange_mutations.sh) (in query: ALTER TABLE test UPDATE test = (SELECT groupArray(id) FROM t1)[n - 99] WHERE 1), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:678: DB::StorageReplicatedMergeTree::waitMutationToFinishOnReplicas(std::vector<String, std::allocator<String>> const&, String const&) const @ 0x00000000380bb8ac
5. ./contrib/llvm-project/libcxx/include/vector:434: DB::StorageReplicatedMergeTree::waitMutation(String const&, unsigned long) const @ 0x00000000382a5d61
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x000000003831cd4c
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
8. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:18:47.431789 [ 3949 ] {6db43001-07f0-4f65-80a0-dcd570e101b7} <Error> TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation '0000000000' with part 'all_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column n): (while reading from part /var/lib/clickhouse/disks/s3/store/2b2/2b26f331-fce6-4e3f-bd41-633e7b761ba3/all_1_1_0/ in table test_13.test (2b26f331-fce6-4e3f-bd41-633e7b761ba3) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 50): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:678: DB::StorageReplicatedMergeTree::waitMutationToFinishOnReplicas(std::vector<String, std::allocator<String>> const&, String const&) const @ 0x00000000380bb8ac
5. ./contrib/llvm-project/libcxx/include/vector:434: DB::StorageReplicatedMergeTree::waitMutation(String const&, unsigned long) const @ 0x00000000382a5d61
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x000000003831cd4c
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
8. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:18:48.472380 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_92_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:49.525796 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:50.419208 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/jey/hopnjkulwxfthxaxrszqjwgyjbnrr, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id bf87802c-9697-48eb-8962-ca53e80ec98d, upload has finished false.
2024.11.04 23:18:50.419471 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/jey/hopnjkulwxfthxaxrszqjwgyjbnrr, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id bf87802c-9697-48eb-8962-ca53e80ec98d, upload has finished false
2024.11.04 23:18:50.571384 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/jey/hopnjkulwxfthxaxrszqjwgyjbnrr, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id bf87802c-9697-48eb-8962-ca53e80ec98d, upload has finished false
2024.11.04 23:18:50.831731 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:50.832167 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 25 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:50.840112 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_92_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:51.056141 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_93_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:52.275551 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {33f0dc8f-1542-43c0-a136-e415d11993b4::202411_1_87_17}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:54.474450 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:54.596361 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_97_20}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:54.612542 [ 7099 ] {e7c3fda9-35b9-4616-a119-321969933ab5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50910) (comment: 01162_strange_mutations.sh) (in query: select count() from t), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:54.618028 [ 7099 ] {e7c3fda9-35b9-4616-a119-321969933ab5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:55.646279 [ 7099 ] {179cc6ca-2c53-4ba2-bde9-719728586aee} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50928) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:55.650105 [ 7099 ] {179cc6ca-2c53-4ba2-bde9-719728586aee} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:18:56.296409 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_97_20}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:56.434926 [ 7099 ] {e7f0d3dc-2983-4bbc-aea0-8a29dcb5d0ce} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50936) (comment: 01162_strange_mutations.sh) (in query: CREATE TABLE test ENGINE=Memory AS SELECT number + 100 AS n, 0 AS test FROM numbers(50)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:56.440832 [ 7099 ] {e7f0d3dc-2983-4bbc-aea0-8a29dcb5d0ce} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:56.766246 [ 7099 ] {0708705c-72c7-47d4-8bfa-91f73ab1a52b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50940) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:56.770340 [ 7099 ] {0708705c-72c7-47d4-8bfa-91f73ab1a52b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:18:56.805433 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/nak/kouxozcjehqvnbgpoporrcynhxtbe, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 682d3b58-5866-4b5c-93e2-b05b29283b22, upload has finished false.
2024.11.04 23:18:56.805603 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/nak/kouxozcjehqvnbgpoporrcynhxtbe, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 682d3b58-5866-4b5c-93e2-b05b29283b22, upload has finished false
2024.11.04 23:18:56.815870 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/nak/kouxozcjehqvnbgpoporrcynhxtbe, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 682d3b58-5866-4b5c-93e2-b05b29283b22, upload has finished false
2024.11.04 23:18:56.820202 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 50 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:57.074470 [ 3352 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:18:57.779600 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_97_20}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationSparse.cpp:167: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cdd498
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:57.779924 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {a7e77ef5-c915-489e-89ce-773df4eb75ed::202411_1_91_18}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:57.800464 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_93_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:57.853537 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 18 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:59.051343 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_97_20}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:18:59.884348 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_97_20}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:01.235127 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_97_20}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:01.398750 [ 7099 ] {5bb5af8c-d014-4fa5-b98f-d38c80e26cc2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51042) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:01.402911 [ 7099 ] {5bb5af8c-d014-4fa5-b98f-d38c80e26cc2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:01.410952 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_93_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:02.269022 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_94_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:02.337250 [ 7099 ] {9c745265-600a-452e-980b-6334bc8e3403} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51056) (comment: 01162_strange_mutations.sh) (in query: select * from t order by n), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:02.347505 [ 7099 ] {9c745265-600a-452e-980b-6334bc8e3403} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:02.472562 [ 3948 ] {761a59c0-6b67-4626-b806-a77264b52cb9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51066) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:02.475718 [ 3948 ] {761a59c0-6b67-4626-b806-a77264b52cb9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:03.859183 [ 3578 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:19:03.859257 [ 3203 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:19:03.859304 [ 3242 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:19:03.859407 [ 3231 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:19:03.859593 [ 3224 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:19:03.859804 [ 3250 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:19:03.859863 [ 3254 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:19:04.530927 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.531227 [ 3258 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.531316 [ 7099 ] {953b986f-3bb4-4db1-b1a0-8cd53e5eb8c4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34290) (comment: 01162_strange_mutations.sh) (in query: select * from t order by n), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:19:04.531518 [ 3274 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.531558 [ 3277 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.531784 [ 3259 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.531962 [ 3255 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8117a
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.532283 [ 3275 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3914: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x0000000038216241
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a61f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.532202 [ 3293 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3914: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x0000000038216241
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a61f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.537540 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_94_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:04.550538 [ 7099 ] {953b986f-3bb4-4db1-b1a0-8cd53e5eb8c4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:19:04.701316 [ 3949 ] {1c3c6c36-6f67-46df-85ec-4bea7b6790a7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34318) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:04.704851 [ 3949 ] {1c3c6c36-6f67-46df-85ec-4bea7b6790a7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:05.080504 [ 3352 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:19:05.087208 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/edi/kerpyiyqfczntqcbimsysybahahco, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d811d90e-a852-4a8f-9592-88ec043eb284, upload has finished false.
2024.11.04 23:19:05.087385 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/edi/kerpyiyqfczntqcbimsysybahahco, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d811d90e-a852-4a8f-9592-88ec043eb284, upload has finished false
2024.11.04 23:19:05.117691 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/edi/kerpyiyqfczntqcbimsysybahahco, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d811d90e-a852-4a8f-9592-88ec043eb284, upload has finished false
2024.11.04 23:19:05.122536 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 56 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:05.214466 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_94_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:06.885817 [ 7099 ] {51156f5d-0396-4130-ac74-386ffa10201b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34366) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:06.889274 [ 7099 ] {51156f5d-0396-4130-ac74-386ffa10201b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:07.161424 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:07.989309 [ 7099 ] {0c7a9b19-2f5a-40d7-8a40-9928ded168ed} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34388) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:19:07.994444 [ 7099 ] {0c7a9b19-2f5a-40d7-8a40-9928ded168ed} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=969). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:19:08.223556 [ 3949 ] {f4a25d36-842a-4636-8433-ffb02c410d47} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=42). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34404) (comment: 01162_strange_mutations.sh) (in query: select count(), sum(n), sum(test) from test), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:08.237235 [ 3949 ] {f4a25d36-842a-4636-8433-ffb02c410d47} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=42, jobs=42). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:09.318913 [ 7099 ] {aee19007-9234-45c8-8744-19e884b2758d} <Error> executeQuery: Code: 48. DB::Exception: Table engine Join supports only DELETE mutations. (NOT_IMPLEMENTED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34420) (comment: 01162_strange_mutations.sh) (in query: ALTER TABLE test UPDATE test = (SELECT groupArray(id) FROM t1)[n - 99] WHERE 1), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Storages/StorageJoin.cpp:122: DB::StorageJoin::checkMutationIsPossible(DB::MutationCommands const&, DB::Settings const&) const @ 0x0000000037d97413
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d5083
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:19:09.322163 [ 7099 ] {aee19007-9234-45c8-8744-19e884b2758d} <Error> TCPHandler: Code: 48. DB::Exception: Table engine Join supports only DELETE mutations. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Storages/StorageJoin.cpp:122: DB::StorageJoin::checkMutationIsPossible(DB::MutationCommands const&, DB::Settings const&) const @ 0x0000000037d97413
4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d5083
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:19:09.764805 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:19:09.765043 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:19:10.303903 [ 7099 ] {bb707a2e-3f56-4777-adeb-c2d5d072b655} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34430) (comment: 01162_strange_mutations.sh) (in query: select count(), sum(n), sum(test) from test), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:10.319704 [ 7099 ] {bb707a2e-3f56-4777-adeb-c2d5d072b655} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:11.024174 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/axe/apfrcxhrqycehlzgonqimxmvvussu, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 639b5d5e-3a5a-41a1-8ee9-11bac267c00e, upload has finished false.
2024.11.04 23:19:11.024535 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/axe/apfrcxhrqycehlzgonqimxmvvussu, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 639b5d5e-3a5a-41a1-8ee9-11bac267c00e, upload has finished false
2024.11.04 23:19:11.044886 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/axe/apfrcxhrqycehlzgonqimxmvvussu, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 639b5d5e-3a5a-41a1-8ee9-11bac267c00e, upload has finished false
2024.11.04 23:19:11.048960 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 23 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:11.323893 [ 7099 ] {3c194c34-fa71-4b98-bd18-030deaa1388f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34462) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:11.327427 [ 7099 ] {3c194c34-fa71-4b98-bd18-030deaa1388f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:11.903239 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_94_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:12.487929 [ 8002 ] {} <Warning> HTTPServerRequest: Got an HTTP request with no content length and no chunked/multipart encoding, it may be impossible to distinguish graceful EOF from abnormal connection loss
2024.11.04 23:19:13.308122 [ 8002 ] {} <Error> Access(user directories): from: ::ffff:127.0.0.1, user: default: Authentication failed: Code: 193. DB::Exception: Invalid credentials. (WRONG_PASSWORD), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Access/IAccessStorage.cpp:763: DB::IAccessStorage::throwInvalidCredentials() @ 0x000000003097182c
4. ./build_docker/./src/Access/IAccessStorage.cpp:0: DB::IAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970c32
5. ./build_docker/./src/Access/IAccessStorage.cpp:511: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030970599
6. ./build_docker/./src/Access/MultipleAccessStorage.cpp:451: DB::MultipleAccessStorage::authenticateImpl(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool, bool) const @ 0x0000000030a365bb
7. ./build_docker/./src/Access/IAccessStorage.cpp:499: DB::IAccessStorage::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, DB::ExternalAuthenticators const&, bool, bool) const @ 0x00000000309703d1
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070f152
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
10. ./build_docker/./src/Server/HTTPHandler.cpp:0: DB::HTTPHandler::authenticateUser(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&) @ 0x000000003a385f46
11. ./build_docker/./src/Server/HTTPHandler.cpp:550: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional<DB::CurrentThread::QueryScope>&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a38a43c
12. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:13.894129 [ 8002 ] {} <Error> DynamicQueryHandler: Code: 516. DB::Exception: default: Authentication failed: password is incorrect, or there is no user with such name.

If you have installed ClickHouse and forgot password you can reset it in the configuration file.
The password for default user is typically located at /etc/clickhouse-server/users.d/default-password.xml
and deleting this file will reset the password.
See also /etc/clickhouse-server/users.xml on the server where ClickHouse is installed.

. (AUTHENTICATION_FAILED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Access/AccessControl.cpp:625: DB::AccessControl::authenticate(DB::Credentials const&, Poco::Net::IPAddress const&, String const&) const @ 0x000000003070fafe
4. ./contrib/llvm-project/libcxx/include/string:1499: DB::Session::authenticate(DB::Credentials const&, Poco::Net::SocketAddress const&) @ 0x0000000035d3bbeb
5. ./build_docker/./src/Server/HTTPHandler.cpp:0: DB::HTTPHandler::authenticateUser(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&) @ 0x000000003a385f46
6. ./build_docker/./src/Server/HTTPHandler.cpp:550: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional<DB::CurrentThread::QueryScope>&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a38a43c
7. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
8. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
12. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
13. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:13.894416 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_95_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:15.306544 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:16.898760 [ 7099 ] {b5e15350-df26-468e-906f-015d024957a8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59648) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:16.902979 [ 7099 ] {b5e15350-df26-468e-906f-015d024957a8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:17.251975 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_95_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:17.608029 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ebm/kkcnbmpfuhxvdjadsielimuftbddn, total size 201326592, count 201326592, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fc5c137a-39ff-45d5-b947-ef823b243811, upload has finished false.
2024.11.04 23:19:17.608436 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ebm/kkcnbmpfuhxvdjadsielimuftbddn, total size 201326592, count 201326592, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fc5c137a-39ff-45d5-b947-ef823b243811, upload has finished false
2024.11.04 23:19:17.652676 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ebm/kkcnbmpfuhxvdjadsielimuftbddn, total size 201326592, count 201326592, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fc5c137a-39ff-45d5-b947-ef823b243811, upload has finished false
2024.11.04 23:19:17.657774 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 9 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:17.793529 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_96_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:18.070633 [ 7099 ] {1352df20-614d-4b76-8e45-aedf31b4b598} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59654) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:18.133642 [ 7099 ] {1352df20-614d-4b76-8e45-aedf31b4b598} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:19.115404 [ 3692 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded hard memory limit (14.54 GiB)
2024.11.04 23:19:19.203737 [ 7099 ] {4f3d3ac9-2e42-41b7-a7e2-0f16ccc45611} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59670) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:19.206891 [ 7099 ] {4f3d3ac9-2e42-41b7-a7e2-0f16ccc45611} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:20.084276 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 91 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:20.263675 [ 7099 ] {fb56bbf3-5f54-479e-bf4f-582f7131dda4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59678) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:20.267324 [ 7099 ] {fb56bbf3-5f54-479e-bf4f-582f7131dda4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:21.727553 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_96_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:21.741513 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_106_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:22.019592 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_106_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:23.601877 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:23.601902 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=59, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:23.602053 [ 3949 ] {4117797e-ff65-4365-a9e4-5a758c0f10f7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:59700) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:23.605818 [ 3949 ] {4117797e-ff65-4365-a9e4-5a758c0f10f7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:23.717875 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 23 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:24.300097 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:24.562027 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:25.190564 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=69, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:25.268882 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_106_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=69, jobs=31). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:26.932051 [ 3949 ] {a56005e1-c87e-477e-b8df-8a1be2208330} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55876) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:26.935519 [ 3949 ] {a56005e1-c87e-477e-b8df-8a1be2208330} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:27.219659 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_96_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=27). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:28.092167 [ 3584 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:19:29.524714 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_106_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=69, jobs=41). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:29.628625 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=69, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:31.345485 [ 3949 ] {35b64012-35bc-4e57-a07e-cc4d19fa9f4a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55916) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:31.349069 [ 3949 ] {35b64012-35bc-4e57-a07e-cc4d19fa9f4a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:32.015143 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_97_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:32.120021 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/xfr/qqoycyzjmyqstiufjfqebdghujijf, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ef45ba8f-0772-42ce-8e6b-d39ee0aa4f25, upload has finished false.
2024.11.04 23:19:32.120172 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/xfr/qqoycyzjmyqstiufjfqebdghujijf, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ef45ba8f-0772-42ce-8e6b-d39ee0aa4f25, upload has finished false
2024.11.04 23:19:32.152552 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/xfr/qqoycyzjmyqstiufjfqebdghujijf, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ef45ba8f-0772-42ce-8e6b-d39ee0aa4f25, upload has finished false
2024.11.04 23:19:32.157676 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 35 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:32.418133 [ 3949 ] {3b9f01d8-58f6-4bb3-a5f1-5efc7389ffad} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:55924) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:32.421592 [ 3949 ] {3b9f01d8-58f6-4bb3-a5f1-5efc7389ffad} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:32.719878 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_106_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=69, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:33.124076 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_97_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:34.324607 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 49 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:36.100038 [ 3584 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:19:36.138059 [ 7099 ] {38e90312-c780-418c-872e-5407328563b0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT BIT_AND(NULL, NULL, NULL);)
2024.11.04 23:19:36.142509 [ 7099 ] {38e90312-c780-418c-872e-5407328563b0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:36.398940 [ 7099 ] {6365ae94-419e-4d6e-a2b8-f39d5fcb8c29} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT BIT_OR(NULL, NULL, NULL);)
2024.11.04 23:19:36.403065 [ 7099 ] {6365ae94-419e-4d6e-a2b8-f39d5fcb8c29} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:36.536603 [ 7099 ] {45a77a21-be3f-4f9f-b17c-2f768b842502} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT BIT_XOR(NULL, NULL, NULL);)
2024.11.04 23:19:36.541227 [ 7099 ] {45a77a21-be3f-4f9f-b17c-2f768b842502} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:36.585628 [ 7099 ] {f777eafa-870d-4d0b-b56f-24f5ace31850} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function BLAKE3 doesn't match: passed 3, should be 1: In scope SELECT BLAKE3(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT BLAKE3(NULL, NULL, NULL);)
2024.11.04 23:19:36.587677 [ 7099 ] {f777eafa-870d-4d0b-b56f-24f5ace31850} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function BLAKE3 doesn't match: passed 3, should be 1: In scope SELECT BLAKE3(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:36.653866 [ 7099 ] {58fe6f99-65f5-46f8-8b4f-24b29fbc6e51} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHARACTER_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT CHARACTER_LENGTH(NULL, NULL, NULL);)
2024.11.04 23:19:36.656076 [ 7099 ] {58fe6f99-65f5-46f8-8b4f-24b29fbc6e51} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHARACTER_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:36.694597 [ 7099 ] {17f99ffc-be50-4079-9a0a-0519621e0064} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHAR_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT CHAR_LENGTH(NULL, NULL, NULL);)
2024.11.04 23:19:36.697333 [ 7099 ] {17f99ffc-be50-4079-9a0a-0519621e0064} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHAR_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:36.928810 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_97_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:36.955106 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 27 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:37.018142 [ 7099 ] {a818bb3e-5f02-4ca8-bdc5-049490f83936} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function CRC32 doesn't match: passed 3, should be 1: In scope SELECT CRC32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT CRC32(NULL, NULL, NULL);)
2024.11.04 23:19:37.020541 [ 7099 ] {a818bb3e-5f02-4ca8-bdc5-049490f83936} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function CRC32 doesn't match: passed 3, should be 1: In scope SELECT CRC32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:37.061788 [ 7099 ] {a746ca9c-9b61-4897-a02c-e8f43138ff43} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function CRC32IEEE doesn't match: passed 3, should be 1: In scope SELECT CRC32IEEE(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT CRC32IEEE(NULL, NULL, NULL);)
2024.11.04 23:19:37.064731 [ 7099 ] {a746ca9c-9b61-4897-a02c-e8f43138ff43} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function CRC32IEEE doesn't match: passed 3, should be 1: In scope SELECT CRC32IEEE(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:37.105926 [ 7099 ] {72b389ed-6e75-423e-ab87-0473cb91f328} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function CRC64 doesn't match: passed 3, should be 1: In scope SELECT CRC64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT CRC64(NULL, NULL, NULL);)
2024.11.04 23:19:37.108315 [ 7099 ] {72b389ed-6e75-423e-ab87-0473cb91f328} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function CRC64 doesn't match: passed 3, should be 1: In scope SELECT CRC64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:37.144707 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_106_21}: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=76, jobs=76). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:37.197572 [ 7099 ] {09e12588-5de2-4459-b3f3-2d2e2babe6d2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT DATABASE(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT DATABASE(NULL, NULL, NULL);)
2024.11.04 23:19:37.200015 [ 7099 ] {09e12588-5de2-4459-b3f3-2d2e2babe6d2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT DATABASE(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:37.308646 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {a7e77ef5-c915-489e-89ce-773df4eb75ed::202411_1_96_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=76, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:37.800066 [ 7099 ] {cdce3279-50bd-4aa4-839d-342db47b8522} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT FORMAT_BYTES(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT FORMAT_BYTES(NULL, NULL, NULL);)
2024.11.04 23:19:37.802522 [ 7099 ] {cdce3279-50bd-4aa4-839d-342db47b8522} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT FORMAT_BYTES(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:37.832413 [ 3949 ] {a2813bd8-e19f-4ab1-8325-402e074d602a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51322) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:37.835743 [ 3949 ] {a2813bd8-e19f-4ab1-8325-402e074d602a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:37.847463 [ 7099 ] {d0a43f37-7259-4527-93f5-2693fd76aef9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT FQDN(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT FQDN(NULL, NULL, NULL);)
2024.11.04 23:19:37.849714 [ 7099 ] {d0a43f37-7259-4527-93f5-2693fd76aef9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT FQDN(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:37.891165 [ 7099 ] {87e7058a-8023-4fa5-90c6-f1f2623bd1c0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT FROM_BASE64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT FROM_BASE64(NULL, NULL, NULL);)
2024.11.04 23:19:37.893259 [ 7099 ] {87e7058a-8023-4fa5-90c6-f1f2623bd1c0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT FROM_BASE64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:37.938307 [ 7099 ] {57227e38-dfd0-4590-95f4-0820f90185b4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT FROM_DAYS(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT FROM_DAYS(NULL, NULL, NULL);)
2024.11.04 23:19:37.940308 [ 7099 ] {57227e38-dfd0-4590-95f4-0820f90185b4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT FROM_DAYS(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.085635 [ 7099 ] {6da4ae06-e9f7-4e19-bdca-f51d5227a8b3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET6_ATON(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT INET6_ATON(NULL, NULL, NULL);)
2024.11.04 23:19:38.087974 [ 7099 ] {6da4ae06-e9f7-4e19-bdca-f51d5227a8b3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET6_ATON(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.202839 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:38.215085 [ 7099 ] {e8331512-6328-46c4-9965-19e89dfafbd4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT INET6_NTOA(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT INET6_NTOA(NULL, NULL, NULL);)
2024.11.04 23:19:38.218694 [ 7099 ] {e8331512-6328-46c4-9965-19e89dfafbd4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT INET6_NTOA(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.343149 [ 7099 ] {5d2c1247-0838-4017-9f0e-ce61fbd2b15a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET_ATON(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT INET_ATON(NULL, NULL, NULL);)
2024.11.04 23:19:38.346674 [ 7099 ] {5d2c1247-0838-4017-9f0e-ce61fbd2b15a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET_ATON(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.374563 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_104_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=76, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:38.393054 [ 7099 ] {553f54ab-3e18-42e8-872c-f89a66c5a9bc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT INET_NTOA(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT INET_NTOA(NULL, NULL, NULL);)
2024.11.04 23:19:38.395539 [ 7099 ] {553f54ab-3e18-42e8-872c-f89a66c5a9bc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT INET_NTOA(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.442513 [ 7099 ] {4113f24b-58dd-471d-ae69-062e0ef62ca8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv4CIDRToRange(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv4CIDRToRange(NULL, NULL, NULL);)
2024.11.04 23:19:38.445188 [ 7099 ] {4113f24b-58dd-471d-ae69-062e0ef62ca8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv4CIDRToRange(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.490721 [ 7099 ] {486cb7e5-a443-45aa-b5f1-006a0d69b0bf} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv4NumToString(NULL, NULL, NULL);)
2024.11.04 23:19:38.492850 [ 7099 ] {486cb7e5-a443-45aa-b5f1-006a0d69b0bf} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.590355 [ 7099 ] {7356044d-afe5-45ee-a81a-91c4f3c63ab1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4NumToStringClassC doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToStringClassC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv4NumToStringClassC(NULL, NULL, NULL);)
2024.11.04 23:19:38.592535 [ 7099 ] {7356044d-afe5-45ee-a81a-91c4f3c63ab1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4NumToStringClassC doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToStringClassC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.790834 [ 7099 ] {0a387048-7107-4c74-9164-37903207f684} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv4StringToNum(NULL, NULL, NULL);)
2024.11.04 23:19:38.793189 [ 7099 ] {0a387048-7107-4c74-9164-37903207f684} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.836543 [ 7099 ] {5b6d96b4-1154-4831-aadc-aa41714f593f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv4StringToNumOrDefault(NULL, NULL, NULL);)
2024.11.04 23:19:38.839216 [ 7099 ] {5b6d96b4-1154-4831-aadc-aa41714f593f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.880318 [ 7099 ] {ddaff178-556b-47af-9250-4d416bf04970} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv4StringToNumOrNull(NULL, NULL, NULL);)
2024.11.04 23:19:38.884630 [ 7099 ] {ddaff178-556b-47af-9250-4d416bf04970} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:38.907472 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=76, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:39.027012 [ 7099 ] {a61ab6de-0df8-4e7d-8a47-7d12fdf3b359} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4ToIPv6 doesn't match: passed 3, should be 1: In scope SELECT IPv4ToIPv6(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv4ToIPv6(NULL, NULL, NULL);)
2024.11.04 23:19:39.029294 [ 7099 ] {a61ab6de-0df8-4e7d-8a47-7d12fdf3b359} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4ToIPv6 doesn't match: passed 3, should be 1: In scope SELECT IPv4ToIPv6(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:39.076113 [ 7099 ] {9253159f-489f-4b69-b2a3-187f081aecd1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv6CIDRToRange(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv6CIDRToRange(NULL, NULL, NULL);)
2024.11.04 23:19:39.078298 [ 7099 ] {9253159f-489f-4b69-b2a3-187f081aecd1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv6CIDRToRange(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:39.118695 [ 7099 ] {64f3c5fa-5bba-487e-8197-d27d9c224695} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv6NumToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv6NumToString(NULL, NULL, NULL);)
2024.11.04 23:19:39.121278 [ 7099 ] {64f3c5fa-5bba-487e-8197-d27d9c224695} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv6NumToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:39.161094 [ 7099 ] {af4ae418-69f0-4f19-9fa6-6d3864bbfd20} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv6StringToNum(NULL, NULL, NULL);)
2024.11.04 23:19:39.163752 [ 7099 ] {af4ae418-69f0-4f19-9fa6-6d3864bbfd20} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:39.234300 [ 7099 ] {bf6e0fa6-1487-4e08-abcf-03bd7414ae39} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv6StringToNumOrDefault(NULL, NULL, NULL);)
2024.11.04 23:19:39.236557 [ 7099 ] {bf6e0fa6-1487-4e08-abcf-03bd7414ae39} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:39.275331 [ 7099 ] {ecdebc15-132d-4ba2-934e-6438be512dd9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT IPv6StringToNumOrNull(NULL, NULL, NULL);)
2024.11.04 23:19:39.277854 [ 7099 ] {ecdebc15-132d-4ba2-934e-6438be512dd9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:39.287419 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_97_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:39.317288 [ 7099 ] {d133e1c0-115d-4242-a1b2-f3b6ded4cf26} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSONArrayLength(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT JSONArrayLength(NULL, NULL, NULL);)
2024.11.04 23:19:39.319468 [ 7099 ] {d133e1c0-115d-4242-a1b2-f3b6ded4cf26} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSONArrayLength(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:39.359888 [ 7099 ] {e7f71932-ef88-468e-a5d3-a865d0afcbd9} <Error> executeQuery: Code: 44. DB::Exception: The last argument of function JSONExtract should be a constant string specifying the return data type, illegal value: NULL: In scope SELECT JSONExtract(NULL, NULL, NULL). (ILLEGAL_COLUMN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT JSONExtract(NULL, NULL, NULL);)
2024.11.04 23:19:39.362589 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:39.362659 [ 7099 ] {e7f71932-ef88-468e-a5d3-a865d0afcbd9} <Error> TCPHandler: Code: 44. DB::Exception: The last argument of function JSONExtract should be a constant string specifying the return data type, illegal value: NULL: In scope SELECT JSONExtract(NULL, NULL, NULL). (ILLEGAL_COLUMN)
2024.11.04 23:19:39.814531 [ 7099 ] {3b025707-0365-4182-8106-a3662c48ef0e} <Error> executeQuery: Code: 44. DB::Exception: The last argument of function JSONExtractKeysAndValues should be a constant string specifying the values' data type, illegal value: NULL: In scope SELECT JSONExtractKeysAndValues(NULL, NULL, NULL). (ILLEGAL_COLUMN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT JSONExtractKeysAndValues(NULL, NULL, NULL);)
2024.11.04 23:19:39.816703 [ 7099 ] {3b025707-0365-4182-8106-a3662c48ef0e} <Error> TCPHandler: Code: 44. DB::Exception: The last argument of function JSONExtractKeysAndValues should be a constant string specifying the values' data type, illegal value: NULL: In scope SELECT JSONExtractKeysAndValues(NULL, NULL, NULL). (ILLEGAL_COLUMN)
2024.11.04 23:19:40.260353 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_97_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:40.313561 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_100_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:40.578934 [ 7099 ] {60b09897-f9eb-4e83-892a-cbe4b974de75} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSON_ARRAY_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT JSON_ARRAY_LENGTH(NULL, NULL, NULL);)
2024.11.04 23:19:40.602697 [ 7099 ] {60b09897-f9eb-4e83-892a-cbe4b974de75} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSON_ARRAY_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:40.981407 [ 7099 ] {be1a97e7-3dd8-46b3-8784-75a1e80e6449} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT L1Distance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L1Distance(NULL, NULL, NULL);)
2024.11.04 23:19:40.983568 [ 7099 ] {be1a97e7-3dd8-46b3-8784-75a1e80e6449} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT L1Distance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.004240 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_100_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:41.022121 [ 7099 ] {27e89867-2207-401e-b491-9cff4ff9cd43} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT L1Norm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L1Norm(NULL, NULL, NULL);)
2024.11.04 23:19:41.024352 [ 7099 ] {27e89867-2207-401e-b491-9cff4ff9cd43} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT L1Norm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.151876 [ 7099 ] {54fae2c5-eda9-4d55-841d-b02eae3676d7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT L1Normalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L1Normalize(NULL, NULL, NULL);)
2024.11.04 23:19:41.154068 [ 7099 ] {54fae2c5-eda9-4d55-841d-b02eae3676d7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT L1Normalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.198377 [ 7099 ] {290ae138-06e5-42f3-8dc6-66384f05617d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT L2Distance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L2Distance(NULL, NULL, NULL);)
2024.11.04 23:19:41.201076 [ 7099 ] {290ae138-06e5-42f3-8dc6-66384f05617d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT L2Distance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.282911 [ 3949 ] {7eb718d4-0730-4277-9955-a20f9757ecca} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51336) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:41.285764 [ 3949 ] {7eb718d4-0730-4277-9955-a20f9757ecca} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:41.294610 [ 7099 ] {c21bee06-1c8c-4ac0-8188-f7f5f0fb1ad2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT L2Norm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L2Norm(NULL, NULL, NULL);)
2024.11.04 23:19:41.296477 [ 7099 ] {c21bee06-1c8c-4ac0-8188-f7f5f0fb1ad2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT L2Norm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.334787 [ 7099 ] {abaf0c66-81fc-4c10-8b2b-4c8ea78e2758} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT L2Normalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L2Normalize(NULL, NULL, NULL);)
2024.11.04 23:19:41.337372 [ 7099 ] {abaf0c66-81fc-4c10-8b2b-4c8ea78e2758} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT L2Normalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.376029 [ 7099 ] {357db6a9-fcdd-4744-a536-eb68a2d612ab} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT L2SquaredDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L2SquaredDistance(NULL, NULL, NULL);)
2024.11.04 23:19:41.421868 [ 7099 ] {357db6a9-fcdd-4744-a536-eb68a2d612ab} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT L2SquaredDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.463217 [ 7099 ] {16d78189-a36a-4962-8a1b-c92125e31c89} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT L2SquaredNorm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT L2SquaredNorm(NULL, NULL, NULL);)
2024.11.04 23:19:41.465107 [ 7099 ] {16d78189-a36a-4962-8a1b-c92125e31c89} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT L2SquaredNorm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.649237 [ 7099 ] {8ccc4895-7de5-4eba-9728-ef0b8710855c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT LinfDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT LinfDistance(NULL, NULL, NULL);)
2024.11.04 23:19:41.651705 [ 7099 ] {8ccc4895-7de5-4eba-9728-ef0b8710855c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT LinfDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.698431 [ 7099 ] {8aa012ce-768d-428d-8630-5e111743fc80} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT LinfNorm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT LinfNorm(NULL, NULL, NULL);)
2024.11.04 23:19:41.700323 [ 7099 ] {8aa012ce-768d-428d-8630-5e111743fc80} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT LinfNorm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.736021 [ 7099 ] {54bde268-c4ab-4f9d-a6ac-51b6edd6ebe3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT LinfNormalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT LinfNormalize(NULL, NULL, NULL);)
2024.11.04 23:19:41.737969 [ 7099 ] {54bde268-c4ab-4f9d-a6ac-51b6edd6ebe3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT LinfNormalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.835215 [ 7099 ] {53876f48-d27a-4af9-9ac4-05e475f2571e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT LpNorm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT LpNorm(NULL, NULL, NULL);)
2024.11.04 23:19:41.837244 [ 7099 ] {53876f48-d27a-4af9-9ac4-05e475f2571e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT LpNorm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.873349 [ 7099 ] {fdb2ce8b-77ac-4c3a-85da-2617e0bec48d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT LpNormalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT LpNormalize(NULL, NULL, NULL);)
2024.11.04 23:19:41.875242 [ 7099 ] {fdb2ce8b-77ac-4c3a-85da-2617e0bec48d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT LpNormalize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:41.912829 [ 7099 ] {c72be668-9b8a-4bad-9ba9-60326e633d77} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function MACNumToString doesn't match: passed 3, should be 1: In scope SELECT MACNumToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT MACNumToString(NULL, NULL, NULL);)
2024.11.04 23:19:41.914933 [ 7099 ] {c72be668-9b8a-4bad-9ba9-60326e633d77} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function MACNumToString doesn't match: passed 3, should be 1: In scope SELECT MACNumToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:42.032944 [ 7099 ] {d376ad99-c760-4003-bf55-0d5b22e1e3ba} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function MACStringToNum doesn't match: passed 3, should be 1: In scope SELECT MACStringToNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT MACStringToNum(NULL, NULL, NULL);)
2024.11.04 23:19:42.035318 [ 7099 ] {d376ad99-c760-4003-bf55-0d5b22e1e3ba} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function MACStringToNum doesn't match: passed 3, should be 1: In scope SELECT MACStringToNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:42.097586 [ 7099 ] {0573778e-7e40-4920-a761-2afd9d41fdab} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function MACStringToOUI doesn't match: passed 3, should be 1: In scope SELECT MACStringToOUI(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT MACStringToOUI(NULL, NULL, NULL);)
2024.11.04 23:19:42.133157 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 97 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:42.178135 [ 7099 ] {0573778e-7e40-4920-a761-2afd9d41fdab} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function MACStringToOUI doesn't match: passed 3, should be 1: In scope SELECT MACStringToOUI(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:42.282852 [ 7099 ] {28234ad9-b6de-40d1-a9d3-af4d47f35367} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT MAP_FROM_ARRAYS(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT MAP_FROM_ARRAYS(NULL, NULL, NULL);)
2024.11.04 23:19:42.285117 [ 7099 ] {28234ad9-b6de-40d1-a9d3-af4d47f35367} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT MAP_FROM_ARRAYS(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:42.323513 [ 7099 ] {6eeb4d91-3819-4a2f-bf36-0009a018ea54} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function MD4 doesn't match: passed 3, should be 1: In scope SELECT MD4(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT MD4(NULL, NULL, NULL);)
2024.11.04 23:19:42.325830 [ 7099 ] {6eeb4d91-3819-4a2f-bf36-0009a018ea54} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function MD4 doesn't match: passed 3, should be 1: In scope SELECT MD4(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:42.345552 [ 3949 ] {5f842eda-0451-413e-87ec-4441c1071650} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51342) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:42.349295 [ 3949 ] {5f842eda-0451-413e-87ec-4441c1071650} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:42.362675 [ 7099 ] {2055e1dc-e3be-4dff-8ef9-013356b77f3c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function MD5 doesn't match: passed 3, should be 1: In scope SELECT MD5(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT MD5(NULL, NULL, NULL);)
2024.11.04 23:19:42.364716 [ 7099 ] {2055e1dc-e3be-4dff-8ef9-013356b77f3c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function MD5 doesn't match: passed 3, should be 1: In scope SELECT MD5(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:42.670638 [ 7099 ] {cf259fe4-fab3-4e21-9d9d-056daee33a82} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT OCTET_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT OCTET_LENGTH(NULL, NULL, NULL);)
2024.11.04 23:19:42.674066 [ 7099 ] {cf259fe4-fab3-4e21-9d9d-056daee33a82} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT OCTET_LENGTH(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.026222 [ 7099 ] {77013b50-736a-4609-8a34-ae52602576a9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT REGEXP_MATCHES(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT REGEXP_MATCHES(NULL, NULL, NULL);)
2024.11.04 23:19:43.032659 [ 7099 ] {77013b50-736a-4609-8a34-ae52602576a9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT REGEXP_MATCHES(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.204416 [ 7099 ] {c19d9759-0b2f-4198-9ad3-c5e95d511785} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT SCHEMA(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SCHEMA(NULL, NULL, NULL);)
2024.11.04 23:19:43.206534 [ 7099 ] {c19d9759-0b2f-4198-9ad3-c5e95d511785} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT SCHEMA(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.319712 [ 7099 ] {bdb9ea80-d0b9-4ba0-b4be-1de0ec834f2e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA1 doesn't match: passed 3, should be 1: In scope SELECT SHA1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SHA1(NULL, NULL, NULL);)
2024.11.04 23:19:43.321939 [ 7099 ] {bdb9ea80-d0b9-4ba0-b4be-1de0ec834f2e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA1 doesn't match: passed 3, should be 1: In scope SELECT SHA1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.360345 [ 7099 ] {279aef50-3aa8-4f79-948d-debf1f0f5ee6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA224 doesn't match: passed 3, should be 1: In scope SELECT SHA224(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SHA224(NULL, NULL, NULL);)
2024.11.04 23:19:43.362446 [ 7099 ] {279aef50-3aa8-4f79-948d-debf1f0f5ee6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA224 doesn't match: passed 3, should be 1: In scope SELECT SHA224(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.400618 [ 7099 ] {bbe5a6f2-da06-4a3b-b630-60312c8b4de8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA256 doesn't match: passed 3, should be 1: In scope SELECT SHA256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SHA256(NULL, NULL, NULL);)
2024.11.04 23:19:43.402918 [ 7099 ] {bbe5a6f2-da06-4a3b-b630-60312c8b4de8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA256 doesn't match: passed 3, should be 1: In scope SELECT SHA256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.440308 [ 7099 ] {be91aabd-c8f4-4453-80bf-de74c6518b21} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA384 doesn't match: passed 3, should be 1: In scope SELECT SHA384(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SHA384(NULL, NULL, NULL);)
2024.11.04 23:19:43.442651 [ 7099 ] {be91aabd-c8f4-4453-80bf-de74c6518b21} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA384 doesn't match: passed 3, should be 1: In scope SELECT SHA384(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.485498 [ 7099 ] {441f7664-a010-412b-95e3-617bb2ab1d5a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA512 doesn't match: passed 3, should be 1: In scope SELECT SHA512(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SHA512(NULL, NULL, NULL);)
2024.11.04 23:19:43.487492 [ 7099 ] {441f7664-a010-412b-95e3-617bb2ab1d5a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA512 doesn't match: passed 3, should be 1: In scope SELECT SHA512(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.524895 [ 7099 ] {dd7e3970-21d8-44bc-a98b-f8b227e6bc82} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA512_256 doesn't match: passed 3, should be 1: In scope SELECT SHA512_256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SHA512_256(NULL, NULL, NULL);)
2024.11.04 23:19:43.527118 [ 7099 ] {dd7e3970-21d8-44bc-a98b-f8b227e6bc82} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA512_256 doesn't match: passed 3, should be 1: In scope SELECT SHA512_256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:43.721942 [ 7099 ] {5880bd79-eedc-4431-91ca-232b6c9d407e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=986). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT STDDEV_POP(NULL, NULL, NULL);)
2024.11.04 23:19:43.725672 [ 7099 ] {5880bd79-eedc-4431-91ca-232b6c9d407e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=986). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:43.822191 [ 7099 ] {f124a5aa-6bef-4341-9963-f9198cbdb91b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT STDDEV_SAMP(NULL, NULL, NULL);)
2024.11.04 23:19:43.826384 [ 7099 ] {f124a5aa-6bef-4341-9963-f9198cbdb91b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:43.931266 [ 7099 ] {b983b362-3cd1-4654-9c71-bdcae0afb33c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=986). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT SVG(NULL, NULL, NULL);)
2024.11.04 23:19:43.933538 [ 7099 ] {b983b362-3cd1-4654-9c71-bdcae0afb33c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=986). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:44.081354 [ 7099 ] {ff79b4fe-5c09-4d1f-a207-ed1f6dbc1ac6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT TO_BASE64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT TO_BASE64(NULL, NULL, NULL);)
2024.11.04 23:19:44.083540 [ 7099 ] {ff79b4fe-5c09-4d1f-a207-ed1f6dbc1ac6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT TO_BASE64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:44.515891 [ 3949 ] {966fc0f8-9712-40a3-a197-fbcb2c54633a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48890) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:44.519551 [ 3949 ] {966fc0f8-9712-40a3-a197-fbcb2c54633a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:44.554391 [ 7099 ] {8825d6b8-1897-4467-b310-104f6c309165} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function URLHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLHierarchy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT URLHierarchy(NULL, NULL, NULL);)
2024.11.04 23:19:44.556782 [ 7099 ] {8825d6b8-1897-4467-b310-104f6c309165} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function URLHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLHierarchy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:44.625586 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_100_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:44.647022 [ 7099 ] {7d79ebcf-ab84-4065-8094-f5b570cb346e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function URLPathHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLPathHierarchy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT URLPathHierarchy(NULL, NULL, NULL);)
2024.11.04 23:19:44.649736 [ 7099 ] {7d79ebcf-ab84-4065-8094-f5b570cb346e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function URLPathHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLPathHierarchy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:44.690062 [ 7099 ] {cc7aecb9-d35d-4fb6-b6b8-0b1eaa2938ad} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTCTimestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT UTCTimestamp(NULL, NULL, NULL);)
2024.11.04 23:19:44.691947 [ 7099 ] {cc7aecb9-d35d-4fb6-b6b8-0b1eaa2938ad} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTCTimestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:44.728904 [ 7099 ] {ca759f2a-27d4-4d82-8289-b6a14aef1676} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTC_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT UTC_timestamp(NULL, NULL, NULL);)
2024.11.04 23:19:44.730959 [ 7099 ] {ca759f2a-27d4-4d82-8289-b6a14aef1676} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTC_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:45.048112 [ 7099 ] {523d844f-1544-4fb1-9498-ace20acd9ad7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT VAR_SAMP(NULL, NULL, NULL);)
2024.11.04 23:19:45.054231 [ 7099 ] {523d844f-1544-4fb1-9498-ace20acd9ad7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:45.566115 [ 7099 ] {2ff5286b-eb38-45be-b1e0-e930633c4402} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function _CAST doesn't match: passed 3, should be 2: In scope SELECT _CAST(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT _CAST(NULL, NULL, NULL);)
2024.11.04 23:19:45.569047 [ 7099 ] {2ff5286b-eb38-45be-b1e0-e930633c4402} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function _CAST doesn't match: passed 3, should be 2: In scope SELECT _CAST(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:45.609705 [ 3949 ] {a6d48cfb-0aa7-4aa7-9b27-13752026ff4e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48902) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:45.611798 [ 7099 ] {efe7ac23-7e77-4555-a238-18858e591edc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function __actionName doesn't match: passed 3, should be 2: In scope SELECT __actionName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT __actionName(NULL, NULL, NULL);)
2024.11.04 23:19:45.612746 [ 3949 ] {a6d48cfb-0aa7-4aa7-9b27-13752026ff4e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:45.614123 [ 7099 ] {efe7ac23-7e77-4555-a238-18858e591edc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function __actionName doesn't match: passed 3, should be 2: In scope SELECT __actionName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:45.737022 [ 7099 ] {d09c247c-614e-4fcb-8bd6-74fa5e7b5a88} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskAnd doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskAnd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT __bitBoolMaskAnd(NULL, NULL, NULL);)
2024.11.04 23:19:45.739038 [ 7099 ] {d09c247c-614e-4fcb-8bd6-74fa5e7b5a88} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskAnd doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskAnd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:45.776713 [ 7099 ] {432ab93e-a2e1-438e-b2f7-7c389de9e670} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskOr doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskOr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT __bitBoolMaskOr(NULL, NULL, NULL);)
2024.11.04 23:19:45.778812 [ 7099 ] {432ab93e-a2e1-438e-b2f7-7c389de9e670} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskOr doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskOr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:45.818383 [ 7099 ] {c3924321-7742-441d-ab39-7297dd58d4bc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitSwapLastTwo doesn't match: passed 3, should be 1: In scope SELECT __bitSwapLastTwo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT __bitSwapLastTwo(NULL, NULL, NULL);)
2024.11.04 23:19:45.820498 [ 7099 ] {c3924321-7742-441d-ab39-7297dd58d4bc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitSwapLastTwo doesn't match: passed 3, should be 1: In scope SELECT __bitSwapLastTwo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:45.948555 [ 7099 ] {8ea220aa-ac5a-44ed-ac67-490ee413f670} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitWrapperFunc doesn't match: passed 3, should be 1: In scope SELECT __bitWrapperFunc(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT __bitWrapperFunc(NULL, NULL, NULL);)
2024.11.04 23:19:45.950688 [ 7099 ] {8ea220aa-ac5a-44ed-ac67-490ee413f670} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitWrapperFunc doesn't match: passed 3, should be 1: In scope SELECT __bitWrapperFunc(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:45.996006 [ 7099 ] {4ef51af5-ed32-48b4-9a37-8616a40c01e8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function __getScalar doesn't match: passed 3, should be 1: In scope SELECT __getScalar(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT __getScalar(NULL, NULL, NULL);)
2024.11.04 23:19:45.998159 [ 7099 ] {4ef51af5-ed32-48b4-9a37-8616a40c01e8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function __getScalar doesn't match: passed 3, should be 1: In scope SELECT __getScalar(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.036657 [ 7099 ] {dcd05212-0261-45aa-b0ed-9d1e5ebe1e35} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function __scalarSubqueryResult doesn't match: passed 3, should be 1: In scope SELECT __scalarSubqueryResult(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT __scalarSubqueryResult(NULL, NULL, NULL);)
2024.11.04 23:19:46.038860 [ 7099 ] {dcd05212-0261-45aa-b0ed-9d1e5ebe1e35} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function __scalarSubqueryResult doesn't match: passed 3, should be 1: In scope SELECT __scalarSubqueryResult(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.076697 [ 7099 ] {f7fbd636-4bb7-492e-a4a2-37e50ea66caa} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function abs doesn't match: passed 3, should be 1: In scope SELECT abs(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT abs(NULL, NULL, NULL);)
2024.11.04 23:19:46.078952 [ 7099 ] {f7fbd636-4bb7-492e-a4a2-37e50ea66caa} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function abs doesn't match: passed 3, should be 1: In scope SELECT abs(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.118124 [ 7099 ] {8bbaa90a-ab02-4ea9-b91f-ef7a30d6970e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function accurateCast doesn't match: passed 3, should be 2: In scope SELECT accurateCast(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT accurateCast(NULL, NULL, NULL);)
2024.11.04 23:19:46.120380 [ 7099 ] {8bbaa90a-ab02-4ea9-b91f-ef7a30d6970e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function accurateCast doesn't match: passed 3, should be 2: In scope SELECT accurateCast(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.159120 [ 7099 ] {930eedec-6c52-4d31-8b51-38d1caf5da76} <Error> executeQuery: Code: 43. DB::Exception: Second argument to accurateCastOrDefault must be a constant string describing type. Actual Nullable(Nothing): In scope SELECT accurateCastOrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT accurateCastOrDefault(NULL, NULL, NULL);)
2024.11.04 23:19:46.161453 [ 7099 ] {930eedec-6c52-4d31-8b51-38d1caf5da76} <Error> TCPHandler: Code: 43. DB::Exception: Second argument to accurateCastOrDefault must be a constant string describing type. Actual Nullable(Nothing): In scope SELECT accurateCastOrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:19:46.198859 [ 7099 ] {fba8aaf8-7628-462c-936e-35b65cd9fc10} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function accurateCastOrNull doesn't match: passed 3, should be 2: In scope SELECT accurateCastOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT accurateCastOrNull(NULL, NULL, NULL);)
2024.11.04 23:19:46.201036 [ 7099 ] {fba8aaf8-7628-462c-936e-35b65cd9fc10} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function accurateCastOrNull doesn't match: passed 3, should be 2: In scope SELECT accurateCastOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.237740 [ 7099 ] {7b321788-a748-4aba-873f-4c59f5bb69b1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function acos doesn't match: passed 3, should be 1: In scope SELECT acos(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT acos(NULL, NULL, NULL);)
2024.11.04 23:19:46.239883 [ 7099 ] {7b321788-a748-4aba-873f-4c59f5bb69b1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function acos doesn't match: passed 3, should be 1: In scope SELECT acos(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.278305 [ 7099 ] {b7f0f300-15fb-486d-bdf9-ba3ba44b7c75} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function acosh doesn't match: passed 3, should be 1: In scope SELECT acosh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT acosh(NULL, NULL, NULL);)
2024.11.04 23:19:46.280827 [ 7099 ] {b7f0f300-15fb-486d-bdf9-ba3ba44b7c75} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function acosh doesn't match: passed 3, should be 1: In scope SELECT acosh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.319043 [ 7099 ] {b11463c4-e2a8-4b32-bf7e-0f69af7e504b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function addDate doesn't match: passed 3, should be 2: In scope SELECT addDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT addDate(NULL, NULL, NULL);)
2024.11.04 23:19:46.321335 [ 7099 ] {b11463c4-e2a8-4b32-bf7e-0f69af7e504b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function addDate doesn't match: passed 3, should be 2: In scope SELECT addDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.545348 [ 7099 ] {fa62227c-9a2a-458f-9981-e16ff664fa80} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function addInterval doesn't match: passed 3, should be 2: In scope SELECT addInterval(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT addInterval(NULL, NULL, NULL);)
2024.11.04 23:19:46.547271 [ 7099 ] {fa62227c-9a2a-458f-9981-e16ff664fa80} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function addInterval doesn't match: passed 3, should be 2: In scope SELECT addInterval(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:46.806805 [ 3949 ] {e7eec11a-0acd-4962-941d-3406d807c819} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48918) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:46.810109 [ 3949 ] {e7eec11a-0acd-4962-941d-3406d807c819} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:47.069458 [ 7099 ] {d83d6f7b-6f2f-437f-b69e-a85ba1b0b34b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function addTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT addTupleOfIntervals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT addTupleOfIntervals(NULL, NULL, NULL);)
2024.11.04 23:19:47.071391 [ 7099 ] {d83d6f7b-6f2f-437f-b69e-a85ba1b0b34b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function addTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT addTupleOfIntervals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:47.080265 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_100_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:47.238614 [ 7099 ] {d40e5c95-082b-484f-b5de-ca69e997d2f5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function addressToLine doesn't match: passed 3, should be 1: In scope SELECT addressToLine(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT addressToLine(NULL, NULL, NULL);)
2024.11.04 23:19:47.240485 [ 7099 ] {d40e5c95-082b-484f-b5de-ca69e997d2f5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function addressToLine doesn't match: passed 3, should be 1: In scope SELECT addressToLine(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:47.277133 [ 7099 ] {5564f85f-3bee-4685-bca3-776d743433e0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function addressToLineWithInlines doesn't match: passed 3, should be 1: In scope SELECT addressToLineWithInlines(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT addressToLineWithInlines(NULL, NULL, NULL);)
2024.11.04 23:19:47.279159 [ 7099 ] {5564f85f-3bee-4685-bca3-776d743433e0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function addressToLineWithInlines doesn't match: passed 3, should be 1: In scope SELECT addressToLineWithInlines(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:47.400471 [ 7099 ] {885a3d4b-f1eb-459b-862e-7901fa4de3c2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function addressToSymbol doesn't match: passed 3, should be 1: In scope SELECT addressToSymbol(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT addressToSymbol(NULL, NULL, NULL);)
2024.11.04 23:19:47.402741 [ 7099 ] {885a3d4b-f1eb-459b-862e-7901fa4de3c2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function addressToSymbol doesn't match: passed 3, should be 1: In scope SELECT addressToSymbol(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:47.927198 [ 7099 ] {92d45811-56b9-4167-bd1e-068509833e2a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT analysisOfVariance(NULL, NULL, NULL);)
2024.11.04 23:19:47.931466 [ 7099 ] {92d45811-56b9-4167-bd1e-068509833e2a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:48.085479 [ 7099 ] {ecf57a9b-097d-4a6c-9ec2-d4f90da00f65} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT anova(NULL, NULL, NULL);)
2024.11.04 23:19:48.090161 [ 7099 ] {ecf57a9b-097d-4a6c-9ec2-d4f90da00f65} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:48.522854 [ 7099 ] {0f91a5b7-9786-4204-b417-200ce0d6b88d} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT anyLast_respect_nulls(NULL, NULL, NULL);)
2024.11.04 23:19:48.525135 [ 7099 ] {0f91a5b7-9786-4204-b417-200ce0d6b88d} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:48.562284 [ 7099 ] {7e8d6df6-c3c8-4dd7-80d9-3e6ba024cca9} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT any_respect_nulls(NULL, NULL, NULL);)
2024.11.04 23:19:48.564839 [ 7099 ] {7e8d6df6-c3c8-4dd7-80d9-3e6ba024cca9} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:48.813023 [ 7099 ] {d89055b4-b415-4bd4-8428-eafda1bc84e8} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT any_value_respect_nulls(NULL, NULL, NULL);)
2024.11.04 23:19:48.815329 [ 7099 ] {d89055b4-b415-4bd4-8428-eafda1bc84e8} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:48.857564 [ 7099 ] {0bde8722-1600-4f8a-a331-dda20f4cb44d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function appendTrailingCharIfAbsent doesn't match: passed 3, should be 2: In scope SELECT appendTrailingCharIfAbsent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT appendTrailingCharIfAbsent(NULL, NULL, NULL);)
2024.11.04 23:19:48.859832 [ 7099 ] {0bde8722-1600-4f8a-a331-dda20f4cb44d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function appendTrailingCharIfAbsent doesn't match: passed 3, should be 2: In scope SELECT appendTrailingCharIfAbsent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:49.185896 [ 3949 ] {4cda48fb-e346-4609-889d-a8379a7af65a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48934) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:49.189210 [ 3949 ] {4cda48fb-e346-4609-889d-a8379a7af65a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:49.298255 [ 7099 ] {c84bddef-9557-4bb4-a6fc-b54bdf311eea} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT approx_top_sum(NULL, NULL, NULL);)
2024.11.04 23:19:49.302473 [ 7099 ] {c84bddef-9557-4bb4-a6fc-b54bdf311eea} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:49.683908 [ 7099 ] {55995735-3336-4d21-9de2-982f81d30eb0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT argMin(NULL, NULL, NULL);)
2024.11.04 23:19:49.689304 [ 7099 ] {55995735-3336-4d21-9de2-982f81d30eb0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:49.786469 [ 7099 ] {e9fe473e-f3de-4d35-9fa0-a0b7725231bd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayAUC doesn't match: passed 3, should be 2: In scope SELECT arrayAUC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayAUC(NULL, NULL, NULL);)
2024.11.04 23:19:49.788805 [ 7099 ] {e9fe473e-f3de-4d35-9fa0-a0b7725231bd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayAUC doesn't match: passed 3, should be 2: In scope SELECT arrayAUC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:49.855196 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_100_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:50.255925 [ 3949 ] {779306c1-090e-42ef-96c5-9334ef270d76} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48950) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:50.259152 [ 3949 ] {779306c1-090e-42ef-96c5-9334ef270d76} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:50.296806 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:50.452774 [ 7099 ] {a1f64ff1-771f-49ff-8940-f166c5ed9be0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayDistinct doesn't match: passed 3, should be 1: In scope SELECT arrayDistinct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayDistinct(NULL, NULL, NULL);)
2024.11.04 23:19:50.454918 [ 7099 ] {a1f64ff1-771f-49ff-8940-f166c5ed9be0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayDistinct doesn't match: passed 3, should be 1: In scope SELECT arrayDistinct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:50.490908 [ 7099 ] {edb2b55d-bde8-4e53-9931-2af912b04434} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayDotProduct doesn't match: passed 3, should be 2: In scope SELECT arrayDotProduct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayDotProduct(NULL, NULL, NULL);)
2024.11.04 23:19:50.493248 [ 7099 ] {edb2b55d-bde8-4e53-9931-2af912b04434} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayDotProduct doesn't match: passed 3, should be 2: In scope SELECT arrayDotProduct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:50.530347 [ 7099 ] {b5a9253e-ae26-4cb6-a5ce-20b4dbedaabb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayElement doesn't match: passed 3, should be 2: In scope SELECT arrayElement(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayElement(NULL, NULL, NULL);)
2024.11.04 23:19:50.532440 [ 7099 ] {b5a9253e-ae26-4cb6-a5ce-20b4dbedaabb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayElement doesn't match: passed 3, should be 2: In scope SELECT arrayElement(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:50.582380 [ 7099 ] {72b7d2dd-90d1-4dbb-ad49-477ffe238a8c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayEnumerate doesn't match: passed 3, should be 1: In scope SELECT arrayEnumerate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayEnumerate(NULL, NULL, NULL);)
2024.11.04 23:19:50.585007 [ 7099 ] {72b7d2dd-90d1-4dbb-ad49-477ffe238a8c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayEnumerate doesn't match: passed 3, should be 1: In scope SELECT arrayEnumerate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:51.232547 [ 7099 ] {3dd18419-932a-422a-b250-6ceebe619868} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=986). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayFirst(NULL, NULL, NULL);)
2024.11.04 23:19:51.235048 [ 7099 ] {3dd18419-932a-422a-b250-6ceebe619868} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=986). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:51.379545 [ 7099 ] {b5d56c9d-f1de-4cdb-9757-5e026d66766e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT arrayFlatten(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayFlatten(NULL, NULL, NULL);)
2024.11.04 23:19:51.381861 [ 7099 ] {b5d56c9d-f1de-4cdb-9757-5e026d66766e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT arrayFlatten(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:51.417156 [ 7099 ] {557a3b59-b7c2-4087-86ed-160b608fa074} <Error> executeQuery: Code: 43. DB::Exception: First argument for function arrayFold must be a function: In scope SELECT arrayFold(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayFold(NULL, NULL, NULL);)
2024.11.04 23:19:51.419022 [ 7099 ] {557a3b59-b7c2-4087-86ed-160b608fa074} <Error> TCPHandler: Code: 43. DB::Exception: First argument for function arrayFold must be a function: In scope SELECT arrayFold(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:19:51.549835 [ 7099 ] {ec70443d-ba2e-4fad-abcd-2d7f8b621e78} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT arrayJaccardIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayJaccardIndex(NULL, NULL, NULL);)
2024.11.04 23:19:51.552182 [ 7099 ] {ec70443d-ba2e-4fad-abcd-2d7f8b621e78} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT arrayJaccardIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:51.700781 [ 7099 ] {3811cc0c-bff9-4983-a2c7-b50d287c3881} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayJoin doesn't match: passed 3, should be 1: In scope SELECT arrayJoin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayJoin(NULL, NULL, NULL);)
2024.11.04 23:19:51.702939 [ 7099 ] {3811cc0c-bff9-4983-a2c7-b50d287c3881} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayJoin doesn't match: passed 3, should be 1: In scope SELECT arrayJoin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.297915 [ 7099 ] {a41ffb99-913c-4704-a4a8-8837c4917ca1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPopBack doesn't match: passed 3, should be 1: In scope SELECT arrayPopBack(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayPopBack(NULL, NULL, NULL);)
2024.11.04 23:19:52.299818 [ 7099 ] {a41ffb99-913c-4704-a4a8-8837c4917ca1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPopBack doesn't match: passed 3, should be 1: In scope SELECT arrayPopBack(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.340687 [ 7099 ] {aeb6defc-f1b1-4044-bedf-7b3e03eba796} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPopFront doesn't match: passed 3, should be 1: In scope SELECT arrayPopFront(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayPopFront(NULL, NULL, NULL);)
2024.11.04 23:19:52.343041 [ 7099 ] {aeb6defc-f1b1-4044-bedf-7b3e03eba796} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPopFront doesn't match: passed 3, should be 1: In scope SELECT arrayPopFront(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.438007 [ 7099 ] {b9ba715d-2c09-4670-a1b9-bd6ff7743aee} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPushBack doesn't match: passed 3, should be 2: In scope SELECT arrayPushBack(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayPushBack(NULL, NULL, NULL);)
2024.11.04 23:19:52.439856 [ 7099 ] {b9ba715d-2c09-4670-a1b9-bd6ff7743aee} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPushBack doesn't match: passed 3, should be 2: In scope SELECT arrayPushBack(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.473474 [ 7099 ] {a4891e75-ba37-424a-a2e4-26fda261da80} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPushFront doesn't match: passed 3, should be 2: In scope SELECT arrayPushFront(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayPushFront(NULL, NULL, NULL);)
2024.11.04 23:19:52.475361 [ 7099 ] {a4891e75-ba37-424a-a2e4-26fda261da80} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPushFront doesn't match: passed 3, should be 2: In scope SELECT arrayPushFront(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.527551 [ 7099 ] {b10c66b2-f66d-4f20-915d-76816e0dc71e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayRandomSample doesn't match: passed 3, should be 2: In scope SELECT arrayRandomSample(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayRandomSample(NULL, NULL, NULL);)
2024.11.04 23:19:52.530069 [ 7099 ] {b10c66b2-f66d-4f20-915d-76816e0dc71e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayRandomSample doesn't match: passed 3, should be 2: In scope SELECT arrayRandomSample(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.565325 [ 7099 ] {b61b74fd-de6e-4fe5-bd36-87b2b6323b18} <Error> executeQuery: Code: 43. DB::Exception: First argument for function arrayReduce must be constant string: name of aggregate function.: In scope SELECT arrayReduce(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayReduce(NULL, NULL, NULL);)
2024.11.04 23:19:52.567194 [ 7099 ] {b61b74fd-de6e-4fe5-bd36-87b2b6323b18} <Error> TCPHandler: Code: 43. DB::Exception: First argument for function arrayReduce must be constant string: name of aggregate function.: In scope SELECT arrayReduce(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:19:52.600448 [ 3949 ] {6a30d90a-83e6-4395-ac41-082151bc9d7b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48966) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:52.603861 [ 3949 ] {6a30d90a-83e6-4395-ac41-082151bc9d7b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:52.707195 [ 7099 ] {118c2112-71fe-4211-aeef-2e7f5d23b0e6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayReverse doesn't match: passed 3, should be 1: In scope SELECT arrayReverse(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayReverse(NULL, NULL, NULL);)
2024.11.04 23:19:52.710624 [ 7099 ] {118c2112-71fe-4211-aeef-2e7f5d23b0e6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayReverse doesn't match: passed 3, should be 1: In scope SELECT arrayReverse(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.853195 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:52.922476 [ 7099 ] {8d0aee81-3cc2-4dc1-bf12-9d2e829d4815} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayRotateLeft doesn't match: passed 3, should be 2: In scope SELECT arrayRotateLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayRotateLeft(NULL, NULL, NULL);)
2024.11.04 23:19:52.924944 [ 7099 ] {8d0aee81-3cc2-4dc1-bf12-9d2e829d4815} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayRotateLeft doesn't match: passed 3, should be 2: In scope SELECT arrayRotateLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:52.961578 [ 7099 ] {e5aabc0d-4bc3-4e1c-9053-6bddc8754bc9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayRotateRight doesn't match: passed 3, should be 2: In scope SELECT arrayRotateRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayRotateRight(NULL, NULL, NULL);)
2024.11.04 23:19:52.963696 [ 7099 ] {e5aabc0d-4bc3-4e1c-9053-6bddc8754bc9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayRotateRight doesn't match: passed 3, should be 2: In scope SELECT arrayRotateRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:53.173668 [ 7099 ] {0ef87495-130a-421b-9bfa-72771063c241} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayShingles doesn't match: passed 3, should be 2: In scope SELECT arrayShingles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayShingles(NULL, NULL, NULL);)
2024.11.04 23:19:53.175773 [ 7099 ] {0ef87495-130a-421b-9bfa-72771063c241} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayShingles doesn't match: passed 3, should be 2: In scope SELECT arrayShingles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:53.662834 [ 3949 ] {ae56efb1-2d26-423a-a0aa-5063667ce4aa} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49964) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:53.712356 [ 3949 ] {ae56efb1-2d26-423a-a0aa-5063667ce4aa} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:53.749273 [ 7099 ] {cbe11676-51d9-442b-9e08-572c8143a07e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayWithConstant doesn't match: passed 3, should be 2: In scope SELECT arrayWithConstant(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT arrayWithConstant(NULL, NULL, NULL);)
2024.11.04 23:19:53.751313 [ 7099 ] {cbe11676-51d9-442b-9e08-572c8143a07e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayWithConstant doesn't match: passed 3, should be 2: In scope SELECT arrayWithConstant(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.089348 [ 7099 ] {94118ff5-b81c-4b40-9687-9975b96c0fab} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function with Array suffix. Must be array. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT array_concat_agg(NULL, NULL, NULL);)
2024.11.04 23:19:54.091148 [ 7099 ] {94118ff5-b81c-4b40-9687-9975b96c0fab} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function with Array suffix. Must be array. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:19:54.104048 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:54.124316 [ 7099 ] {108fb051-c3fb-4342-8e48-8e6793a036a6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ascii doesn't match: passed 3, should be 1: In scope SELECT ascii(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ascii(NULL, NULL, NULL);)
2024.11.04 23:19:54.126107 [ 7099 ] {108fb051-c3fb-4342-8e48-8e6793a036a6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ascii doesn't match: passed 3, should be 1: In scope SELECT ascii(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.206436 [ 7099 ] {f7b348cf-89dc-48e8-8f6d-62810ce5d8a7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function asin doesn't match: passed 3, should be 1: In scope SELECT asin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT asin(NULL, NULL, NULL);)
2024.11.04 23:19:54.208539 [ 7099 ] {f7b348cf-89dc-48e8-8f6d-62810ce5d8a7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function asin doesn't match: passed 3, should be 1: In scope SELECT asin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.243160 [ 7099 ] {f039b446-6b56-4a2f-b6c7-2a4831acc08c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function asinh doesn't match: passed 3, should be 1: In scope SELECT asinh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT asinh(NULL, NULL, NULL);)
2024.11.04 23:19:54.245227 [ 7099 ] {f039b446-6b56-4a2f-b6c7-2a4831acc08c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function asinh doesn't match: passed 3, should be 1: In scope SELECT asinh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.282176 [ 7099 ] {635b245a-e263-412f-a167-8aaf862e40cf} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function assumeNotNull doesn't match: passed 3, should be 1: In scope SELECT assumeNotNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT assumeNotNull(NULL, NULL, NULL);)
2024.11.04 23:19:54.284333 [ 7099 ] {635b245a-e263-412f-a167-8aaf862e40cf} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function assumeNotNull doesn't match: passed 3, should be 1: In scope SELECT assumeNotNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.319966 [ 7099 ] {943d1748-212b-4816-a7d5-78157a886651} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function atan doesn't match: passed 3, should be 1: In scope SELECT atan(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT atan(NULL, NULL, NULL);)
2024.11.04 23:19:54.322291 [ 7099 ] {943d1748-212b-4816-a7d5-78157a886651} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function atan doesn't match: passed 3, should be 1: In scope SELECT atan(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.357248 [ 7099 ] {55f95305-a2d9-4425-8b48-ee4711e94086} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function atan2 doesn't match: passed 3, should be 2: In scope SELECT atan2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT atan2(NULL, NULL, NULL);)
2024.11.04 23:19:54.359397 [ 7099 ] {55f95305-a2d9-4425-8b48-ee4711e94086} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function atan2 doesn't match: passed 3, should be 2: In scope SELECT atan2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.393893 [ 7099 ] {577ed8b2-f34a-4b26-8372-39587f8792c4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function atanh doesn't match: passed 3, should be 1: In scope SELECT atanh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT atanh(NULL, NULL, NULL);)
2024.11.04 23:19:54.395824 [ 7099 ] {577ed8b2-f34a-4b26-8372-39587f8792c4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function atanh doesn't match: passed 3, should be 1: In scope SELECT atanh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.613847 [ 7099 ] {0b38a4cd-0125-4772-93ea-5dab7e16fa9d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=33, jobs=33). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT avgWeighted(NULL, NULL, NULL);)
2024.11.04 23:19:54.618103 [ 7099 ] {0b38a4cd-0125-4772-93ea-5dab7e16fa9d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=33, jobs=33). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:54.702308 [ 7099 ] {83baebca-792f-436f-b8d0-1b2bce7a93d3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function base58Decode doesn't match: passed 3, should be 1: In scope SELECT base58Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT base58Decode(NULL, NULL, NULL);)
2024.11.04 23:19:54.705953 [ 7099 ] {83baebca-792f-436f-b8d0-1b2bce7a93d3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function base58Decode doesn't match: passed 3, should be 1: In scope SELECT base58Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.740813 [ 7099 ] {f3832e6d-9590-43ec-9484-4292a0656571} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function base58Encode doesn't match: passed 3, should be 1: In scope SELECT base58Encode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT base58Encode(NULL, NULL, NULL);)
2024.11.04 23:19:54.742841 [ 7099 ] {f3832e6d-9590-43ec-9484-4292a0656571} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function base58Encode doesn't match: passed 3, should be 1: In scope SELECT base58Encode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.777425 [ 7099 ] {7cebeb84-97e9-4bd7-a3a9-2837f8401018} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT base64Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT base64Decode(NULL, NULL, NULL);)
2024.11.04 23:19:54.780002 [ 7099 ] {7cebeb84-97e9-4bd7-a3a9-2837f8401018} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT base64Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.814234 [ 7099 ] {06e0af3f-65f4-4ebf-8069-4af7e8ba5904} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT base64Encode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT base64Encode(NULL, NULL, NULL);)
2024.11.04 23:19:54.816101 [ 7099 ] {06e0af3f-65f4-4ebf-8069-4af7e8ba5904} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT base64Encode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.823491 [ 3949 ] {a28ccd1f-917b-475b-a527-f985205b0fb0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49980) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:54.826757 [ 3949 ] {a28ccd1f-917b-475b-a527-f985205b0fb0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:54.850193 [ 7099 ] {6b4fb9c5-e6d2-46cb-b0b5-631f1b44d30e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function basename doesn't match: passed 3, should be 1: In scope SELECT basename(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT basename(NULL, NULL, NULL);)
2024.11.04 23:19:54.852636 [ 7099 ] {6b4fb9c5-e6d2-46cb-b0b5-631f1b44d30e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function basename doesn't match: passed 3, should be 1: In scope SELECT basename(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.888181 [ 7099 ] {7d195432-b7b1-4dfe-9dd1-b1f9aca302ef} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bin doesn't match: passed 3, should be 1: In scope SELECT bin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bin(NULL, NULL, NULL);)
2024.11.04 23:19:54.890218 [ 7099 ] {7d195432-b7b1-4dfe-9dd1-b1f9aca302ef} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bin doesn't match: passed 3, should be 1: In scope SELECT bin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.924589 [ 7099 ] {4caac2f3-6dc0-4083-a162-bff0e080a1d5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitAnd doesn't match: passed 3, should be 2: In scope SELECT bitAnd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitAnd(NULL, NULL, NULL);)
2024.11.04 23:19:54.926510 [ 7099 ] {4caac2f3-6dc0-4083-a162-bff0e080a1d5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitAnd doesn't match: passed 3, should be 2: In scope SELECT bitAnd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.966120 [ 7099 ] {1af29e12-1600-4578-af85-ef8d121cfc43} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitCount doesn't match: passed 3, should be 1: In scope SELECT bitCount(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitCount(NULL, NULL, NULL);)
2024.11.04 23:19:54.968366 [ 7099 ] {1af29e12-1600-4578-af85-ef8d121cfc43} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitCount doesn't match: passed 3, should be 1: In scope SELECT bitCount(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:54.972748 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/uij/xpanhgoicdzgpgvrslwuwfjelvnpo, total size 452984832, count 452984832, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 576b420f-5a3d-43f2-8f16-5cb053cba8bf, upload has finished false.
2024.11.04 23:19:54.972894 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/uij/xpanhgoicdzgpgvrslwuwfjelvnpo, total size 452984832, count 452984832, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 576b420f-5a3d-43f2-8f16-5cb053cba8bf, upload has finished false
2024.11.04 23:19:55.008781 [ 7099 ] {8f81c1cf-43cd-43c9-a716-5fbe6ac05315} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitHammingDistance doesn't match: passed 3, should be 2: In scope SELECT bitHammingDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitHammingDistance(NULL, NULL, NULL);)
2024.11.04 23:19:55.010924 [ 7099 ] {8f81c1cf-43cd-43c9-a716-5fbe6ac05315} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitHammingDistance doesn't match: passed 3, should be 2: In scope SELECT bitHammingDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.044033 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/uij/xpanhgoicdzgpgvrslwuwfjelvnpo, total size 452984832, count 452984832, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 576b420f-5a3d-43f2-8f16-5cb053cba8bf, upload has finished false
2024.11.04 23:19:55.046451 [ 7099 ] {c4553160-f3c5-48c0-987a-5ded0565cfcf} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitNot doesn't match: passed 3, should be 1: In scope SELECT bitNot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitNot(NULL, NULL, NULL);)
2024.11.04 23:19:55.048405 [ 7099 ] {c4553160-f3c5-48c0-987a-5ded0565cfcf} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitNot doesn't match: passed 3, should be 1: In scope SELECT bitNot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.142656 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_8_8_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 58 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:55.178121 [ 7099 ] {dd846032-ae88-4126-8127-c1899429b0d4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitOr doesn't match: passed 3, should be 2: In scope SELECT bitOr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitOr(NULL, NULL, NULL);)
2024.11.04 23:19:55.180116 [ 7099 ] {dd846032-ae88-4126-8127-c1899429b0d4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitOr doesn't match: passed 3, should be 2: In scope SELECT bitOr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.216711 [ 7099 ] {96aa747f-5800-4440-874a-9848bc91727c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitPositionsToArray doesn't match: passed 3, should be 1: In scope SELECT bitPositionsToArray(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitPositionsToArray(NULL, NULL, NULL);)
2024.11.04 23:19:55.218987 [ 7099 ] {96aa747f-5800-4440-874a-9848bc91727c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitPositionsToArray doesn't match: passed 3, should be 1: In scope SELECT bitPositionsToArray(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.253813 [ 7099 ] {9bba9b82-36be-45ce-82a1-47f87f5fa055} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitRotateLeft doesn't match: passed 3, should be 2: In scope SELECT bitRotateLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitRotateLeft(NULL, NULL, NULL);)
2024.11.04 23:19:55.255865 [ 7099 ] {9bba9b82-36be-45ce-82a1-47f87f5fa055} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitRotateLeft doesn't match: passed 3, should be 2: In scope SELECT bitRotateLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.298899 [ 7099 ] {a58599b0-1285-48b5-b7d8-037c9513121b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitRotateRight doesn't match: passed 3, should be 2: In scope SELECT bitRotateRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitRotateRight(NULL, NULL, NULL);)
2024.11.04 23:19:55.301005 [ 7099 ] {a58599b0-1285-48b5-b7d8-037c9513121b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitRotateRight doesn't match: passed 3, should be 2: In scope SELECT bitRotateRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.336372 [ 7099 ] {56cb757d-b8e8-41f2-8fdb-fd6c856c4d78} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitShiftLeft doesn't match: passed 3, should be 2: In scope SELECT bitShiftLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitShiftLeft(NULL, NULL, NULL);)
2024.11.04 23:19:55.338256 [ 7099 ] {56cb757d-b8e8-41f2-8fdb-fd6c856c4d78} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitShiftLeft doesn't match: passed 3, should be 2: In scope SELECT bitShiftLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.371117 [ 7099 ] {ee60b6bc-3681-4a6e-b96c-abc5eb21c4bc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitShiftRight doesn't match: passed 3, should be 2: In scope SELECT bitShiftRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitShiftRight(NULL, NULL, NULL);)
2024.11.04 23:19:55.373174 [ 7099 ] {ee60b6bc-3681-4a6e-b96c-abc5eb21c4bc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitShiftRight doesn't match: passed 3, should be 2: In scope SELECT bitShiftRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.592198 [ 7099 ] {fd7e10dd-48ba-4f11-8743-fa3a32456073} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitTest doesn't match: passed 3, should be 2: In scope SELECT bitTest(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitTest(NULL, NULL, NULL);)
2024.11.04 23:19:55.594444 [ 7099 ] {fd7e10dd-48ba-4f11-8743-fa3a32456073} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitTest doesn't match: passed 3, should be 2: In scope SELECT bitTest(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.873226 [ 7099 ] {f124fa2c-edef-4483-8225-1ddec8514d66} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitXor doesn't match: passed 3, should be 2: In scope SELECT bitXor(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitXor(NULL, NULL, NULL);)
2024.11.04 23:19:55.875569 [ 7099 ] {f124fa2c-edef-4483-8225-1ddec8514d66} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitXor doesn't match: passed 3, should be 2: In scope SELECT bitXor(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.891095 [ 3949 ] {824852d0-60a7-4b34-b379-c9ca84b83d93} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49994) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:55.894666 [ 3949 ] {824852d0-60a7-4b34-b379-c9ca84b83d93} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:55.915213 [ 7099 ] {8e5cbd81-b0b4-4295-83e7-3892f1a25ca7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAnd doesn't match: passed 3, should be 2: In scope SELECT bitmapAnd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapAnd(NULL, NULL, NULL);)
2024.11.04 23:19:55.917702 [ 7099 ] {8e5cbd81-b0b4-4295-83e7-3892f1a25ca7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAnd doesn't match: passed 3, should be 2: In scope SELECT bitmapAnd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:55.954694 [ 7099 ] {e2c93907-7283-4b95-b482-fd5b0a5d1380} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAndCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapAndCardinality(NULL, NULL, NULL);)
2024.11.04 23:19:56.036840 [ 7099 ] {e2c93907-7283-4b95-b482-fd5b0a5d1380} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAndCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.080489 [ 7099 ] {c8186cb9-7658-4053-a52d-52e6d26c884a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAndnot doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapAndnot(NULL, NULL, NULL);)
2024.11.04 23:19:56.083366 [ 7099 ] {c8186cb9-7658-4053-a52d-52e6d26c884a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAndnot doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.123803 [ 7099 ] {260f73ba-be4b-43e7-b47b-e9d11db705d1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAndnotCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnotCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapAndnotCardinality(NULL, NULL, NULL);)
2024.11.04 23:19:56.126026 [ 7099 ] {260f73ba-be4b-43e7-b47b-e9d11db705d1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAndnotCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnotCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.160782 [ 7099 ] {829061af-6d75-4514-a698-f88ce3c1aeea} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapBuild doesn't match: passed 3, should be 1: In scope SELECT bitmapBuild(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapBuild(NULL, NULL, NULL);)
2024.11.04 23:19:56.162747 [ 7099 ] {829061af-6d75-4514-a698-f88ce3c1aeea} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapBuild doesn't match: passed 3, should be 1: In scope SELECT bitmapBuild(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.200952 [ 7099 ] {abb06a1a-da6a-4861-bb1c-597e2c2ce933} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapCardinality doesn't match: passed 3, should be 1: In scope SELECT bitmapCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapCardinality(NULL, NULL, NULL);)
2024.11.04 23:19:56.203592 [ 7099 ] {abb06a1a-da6a-4861-bb1c-597e2c2ce933} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapCardinality doesn't match: passed 3, should be 1: In scope SELECT bitmapCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.240581 [ 7099 ] {fca281d5-bce4-4770-8fc7-876f84141bac} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapContains doesn't match: passed 3, should be 2: In scope SELECT bitmapContains(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapContains(NULL, NULL, NULL);)
2024.11.04 23:19:56.242608 [ 7099 ] {fca281d5-bce4-4770-8fc7-876f84141bac} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapContains doesn't match: passed 3, should be 2: In scope SELECT bitmapContains(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.278026 [ 7099 ] {a8261a64-6cfc-406d-9d62-ec3ae0400395} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapHasAll doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAll(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapHasAll(NULL, NULL, NULL);)
2024.11.04 23:19:56.280128 [ 7099 ] {a8261a64-6cfc-406d-9d62-ec3ae0400395} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapHasAll doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAll(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.315614 [ 7099 ] {55313ba7-4232-4a69-b421-8c890ed7c413} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapHasAny doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapHasAny(NULL, NULL, NULL);)
2024.11.04 23:19:56.317759 [ 7099 ] {55313ba7-4232-4a69-b421-8c890ed7c413} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapHasAny doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.401317 [ 7099 ] {151e80cc-f059-4bb4-9447-2589c783f626} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapMax doesn't match: passed 3, should be 1: In scope SELECT bitmapMax(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapMax(NULL, NULL, NULL);)
2024.11.04 23:19:56.403784 [ 7099 ] {151e80cc-f059-4bb4-9447-2589c783f626} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapMax doesn't match: passed 3, should be 1: In scope SELECT bitmapMax(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.451293 [ 7099 ] {5d24c670-7b6a-4c9a-b3a1-8793938f271d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapMin doesn't match: passed 3, should be 1: In scope SELECT bitmapMin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapMin(NULL, NULL, NULL);)
2024.11.04 23:19:56.453353 [ 7099 ] {5d24c670-7b6a-4c9a-b3a1-8793938f271d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapMin doesn't match: passed 3, should be 1: In scope SELECT bitmapMin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.489722 [ 7099 ] {2c361e3f-5d7b-4e85-91fc-687e508baedb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapOr doesn't match: passed 3, should be 2: In scope SELECT bitmapOr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapOr(NULL, NULL, NULL);)
2024.11.04 23:19:56.491901 [ 7099 ] {2c361e3f-5d7b-4e85-91fc-687e508baedb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapOr doesn't match: passed 3, should be 2: In scope SELECT bitmapOr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.532054 [ 7099 ] {2ab9f992-dfbc-4572-ae9a-63748f98de0e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapOrCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapOrCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapOrCardinality(NULL, NULL, NULL);)
2024.11.04 23:19:56.534607 [ 7099 ] {2ab9f992-dfbc-4572-ae9a-63748f98de0e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapOrCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapOrCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.695648 [ 7099 ] {5522f575-7a0f-4194-900d-fba66cd4729e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapToArray doesn't match: passed 3, should be 1: In scope SELECT bitmapToArray(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapToArray(NULL, NULL, NULL);)
2024.11.04 23:19:56.698021 [ 7099 ] {5522f575-7a0f-4194-900d-fba66cd4729e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapToArray doesn't match: passed 3, should be 1: In scope SELECT bitmapToArray(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.780938 [ 7099 ] {7669fdcd-f468-4f50-8cbf-0b7a5b24a1d7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapXor doesn't match: passed 3, should be 2: In scope SELECT bitmapXor(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapXor(NULL, NULL, NULL);)
2024.11.04 23:19:56.782756 [ 7099 ] {7669fdcd-f468-4f50-8cbf-0b7a5b24a1d7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapXor doesn't match: passed 3, should be 2: In scope SELECT bitmapXor(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.872642 [ 7099 ] {3aae35da-38f5-40cd-bad6-3a4804f79480} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapXorCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapXorCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmapXorCardinality(NULL, NULL, NULL);)
2024.11.04 23:19:56.876731 [ 7099 ] {3aae35da-38f5-40cd-bad6-3a4804f79480} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapXorCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapXorCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.917799 [ 7099 ] {829e2fad-44fe-4e03-9b5f-4d2de0553cfd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmaskToArray doesn't match: passed 3, should be 1: In scope SELECT bitmaskToArray(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmaskToArray(NULL, NULL, NULL);)
2024.11.04 23:19:56.919900 [ 7099 ] {829e2fad-44fe-4e03-9b5f-4d2de0553cfd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmaskToArray doesn't match: passed 3, should be 1: In scope SELECT bitmaskToArray(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.955264 [ 7099 ] {92bffbab-c8da-4a5d-96c5-c5982366bf0c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmaskToList doesn't match: passed 3, should be 1: In scope SELECT bitmaskToList(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT bitmaskToList(NULL, NULL, NULL);)
2024.11.04 23:19:56.957613 [ 7099 ] {92bffbab-c8da-4a5d-96c5-c5982366bf0c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmaskToList doesn't match: passed 3, should be 1: In scope SELECT bitmaskToList(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:56.977414 [ 3949 ] {71f91cd9-4a8f-41a8-bffa-9952a4a89e44} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49996) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:56.982988 [ 3949 ] {71f91cd9-4a8f-41a8-bffa-9952a4a89e44} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:56.994215 [ 7099 ] {0cff5829-d41c-4ece-99a8-f1da5aa8bff8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function blockNumber doesn't match: passed 3, should be 0: In scope SELECT blockNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT blockNumber(NULL, NULL, NULL);)
2024.11.04 23:19:56.996722 [ 7099 ] {0cff5829-d41c-4ece-99a8-f1da5aa8bff8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function blockNumber doesn't match: passed 3, should be 0: In scope SELECT blockNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:57.085076 [ 7099 ] {fb87b70d-ea4c-40f7-bb2b-34d2bd0c5873} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function blockSize doesn't match: passed 3, should be 0: In scope SELECT blockSize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT blockSize(NULL, NULL, NULL);)
2024.11.04 23:19:57.087021 [ 7099 ] {fb87b70d-ea4c-40f7-bb2b-34d2bd0c5873} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function blockSize doesn't match: passed 3, should be 0: In scope SELECT blockSize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:57.277539 [ 7099 ] {ce5b12cb-489d-406b-96b9-ba632494a4af} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function buildId doesn't match: passed 3, should be 0: In scope SELECT buildId(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT buildId(NULL, NULL, NULL);)
2024.11.04 23:19:57.279861 [ 7099 ] {ce5b12cb-489d-406b-96b9-ba632494a4af} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function buildId doesn't match: passed 3, should be 0: In scope SELECT buildId(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:57.319530 [ 7099 ] {8f7a7568-d8de-4229-b34e-080f8af43e24} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT byteHammingDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT byteHammingDistance(NULL, NULL, NULL);)
2024.11.04 23:19:57.321910 [ 7099 ] {8f7a7568-d8de-4229-b34e-080f8af43e24} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT byteHammingDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:57.369072 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:19:57.571467 [ 7099 ] {26bf0d3e-bcb6-4dcc-8586-e2618ea870b7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function byteSwap doesn't match: passed 3, should be 1: In scope SELECT byteSwap(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT byteSwap(NULL, NULL, NULL);)
2024.11.04 23:19:57.573522 [ 7099 ] {26bf0d3e-bcb6-4dcc-8586-e2618ea870b7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function byteSwap doesn't match: passed 3, should be 1: In scope SELECT byteSwap(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:57.842678 [ 7099 ] {7720eadf-0fbe-4dc0-86e6-9082a50f191b} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function catboostEvaluate, expected a string.: In scope SELECT catboostEvaluate(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT catboostEvaluate(NULL, NULL, NULL);)
2024.11.04 23:19:57.844803 [ 7099 ] {7720eadf-0fbe-4dc0-86e6-9082a50f191b} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function catboostEvaluate, expected a string.: In scope SELECT catboostEvaluate(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:19:57.878597 [ 7099 ] {eb3bea44-cf66-4fb9-9171-bd69cb4c737f} <Error> executeQuery: Code: 43. DB::Exception: All the arguments of aggregate function categoricalInformationValue should be UInt8. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT categoricalInformationValue(NULL, NULL, NULL);)
2024.11.04 23:19:57.880641 [ 7099 ] {eb3bea44-cf66-4fb9-9171-bd69cb4c737f} <Error> TCPHandler: Code: 43. DB::Exception: All the arguments of aggregate function categoricalInformationValue should be UInt8. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:19:57.914416 [ 7099 ] {46c6c6c5-f01c-4ae3-a70c-437b8ae82c8b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cbrt doesn't match: passed 3, should be 1: In scope SELECT cbrt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cbrt(NULL, NULL, NULL);)
2024.11.04 23:19:57.916387 [ 7099 ] {46c6c6c5-f01c-4ae3-a70c-437b8ae82c8b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cbrt doesn't match: passed 3, should be 1: In scope SELECT cbrt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:58.133391 [ 3949 ] {0771d67a-d0a8-4474-8106-4f0888cdab13} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50004) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:58.137024 [ 3949 ] {0771d67a-d0a8-4474-8106-4f0888cdab13} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:19:58.706044 [ 7099 ] {5ea0ac4d-e737-4b2f-9449-9d5e49f2422e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connectionId(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT connectionId(NULL, NULL, NULL);)
2024.11.04 23:19:58.708003 [ 7099 ] {5ea0ac4d-e737-4b2f-9449-9d5e49f2422e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connectionId(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:58.742452 [ 7099 ] {2eb6d2b8-443b-4ddb-914b-fa9d045be564} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connection_id(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT connection_id(NULL, NULL, NULL);)
2024.11.04 23:19:58.744452 [ 7099 ] {2eb6d2b8-443b-4ddb-914b-fa9d045be564} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connection_id(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.109054 [ 3582 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:19:59.187476 [ 7099 ] {f10817cc-069d-4b14-a40c-8c9a6a29a1b8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT corrMatrix(NULL, NULL, NULL);)
2024.11.04 23:19:59.191481 [ 7099 ] {f10817cc-069d-4b14-a40c-8c9a6a29a1b8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:19:59.235971 [ 3949 ] {67163271-6939-4ee0-86e2-1d8e41b5415b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50010) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:59.238840 [ 3949 ] {67163271-6939-4ee0-86e2-1d8e41b5415b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:19:59.363169 [ 7099 ] {42396b77-fe80-422a-afc1-779af6a9183d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cos doesn't match: passed 3, should be 1: In scope SELECT cos(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cos(NULL, NULL, NULL);)
2024.11.04 23:19:59.365363 [ 7099 ] {42396b77-fe80-422a-afc1-779af6a9183d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cos doesn't match: passed 3, should be 1: In scope SELECT cos(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.401306 [ 7099 ] {fa75a81a-2cde-4dab-9306-da274df24d68} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cosh doesn't match: passed 3, should be 1: In scope SELECT cosh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cosh(NULL, NULL, NULL);)
2024.11.04 23:19:59.403669 [ 7099 ] {fa75a81a-2cde-4dab-9306-da274df24d68} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cosh doesn't match: passed 3, should be 1: In scope SELECT cosh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.454885 [ 7099 ] {4699e636-8463-4ce6-a7ce-1f3cf17784aa} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cosineDistance doesn't match: passed 3, should be 2: In scope SELECT cosineDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cosineDistance(NULL, NULL, NULL);)
2024.11.04 23:19:59.457050 [ 7099 ] {4699e636-8463-4ce6-a7ce-1f3cf17784aa} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cosineDistance doesn't match: passed 3, should be 2: In scope SELECT cosineDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.491169 [ 7099 ] {003a3d58-2de6-4fc4-be60-33252d57fc6b} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function count requires zero or one argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT count(NULL, NULL, NULL);)
2024.11.04 23:19:59.493365 [ 7099 ] {003a3d58-2de6-4fc4-be60-33252d57fc6b} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function count requires zero or one argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.527749 [ 7099 ] {0bcfd578-4647-4456-8524-d40c49ee8097} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function countDigits doesn't match: passed 3, should be 1: In scope SELECT countDigits(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT countDigits(NULL, NULL, NULL);)
2024.11.04 23:19:59.530149 [ 7099 ] {0bcfd578-4647-4456-8524-d40c49ee8097} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function countDigits doesn't match: passed 3, should be 1: In scope SELECT countDigits(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.564954 [ 7099 ] {9508f40f-5efd-4ef1-a6ff-8757fd5b53e9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function countEqual doesn't match: passed 3, should be 2: In scope SELECT countEqual(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT countEqual(NULL, NULL, NULL);)
2024.11.04 23:19:59.567103 [ 7099 ] {9508f40f-5efd-4ef1-a6ff-8757fd5b53e9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function countEqual doesn't match: passed 3, should be 2: In scope SELECT countEqual(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.600211 [ 7099 ] {34ea7a76-2052-42a1-a591-d15a4de82db5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function countMatches doesn't match: passed 3, should be 2: In scope SELECT countMatches(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT countMatches(NULL, NULL, NULL);)
2024.11.04 23:19:59.602104 [ 7099 ] {34ea7a76-2052-42a1-a591-d15a4de82db5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function countMatches doesn't match: passed 3, should be 2: In scope SELECT countMatches(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.635564 [ 7099 ] {b1970b74-28ea-4c39-8b5c-f2c5e6ebe0dc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function countMatchesCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT countMatchesCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT countMatchesCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:19:59.703091 [ 7099 ] {b1970b74-28ea-4c39-8b5c-f2c5e6ebe0dc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function countMatchesCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT countMatchesCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:19:59.852601 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=76, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:00.031784 [ 7099 ] {30afd7c5-5574-4c06-93cf-edc797a002ec} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT covarPop(NULL, NULL, NULL);)
2024.11.04 23:20:00.036111 [ 7099 ] {30afd7c5-5574-4c06-93cf-edc797a002ec} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:00.251607 [ 7099 ] {855d49a4-561d-4fea-89ed-43b46623c08a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT covarPopStable(NULL, NULL, NULL);)
2024.11.04 23:20:00.316726 [ 7099 ] {855d49a4-561d-4fea-89ed-43b46623c08a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:00.771526 [ 7099 ] {20a47f0b-e660-4687-872d-87783187768c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=30, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT covarSampMatrix(NULL, NULL, NULL);)
2024.11.04 23:20:00.776958 [ 7099 ] {20a47f0b-e660-4687-872d-87783187768c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=30, jobs=2). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:01.214994 [ 7099 ] {08c0bb4f-a329-4e58-9ceb-f712c91b17d3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cramersV(NULL, NULL, NULL);)
2024.11.04 23:20:01.219267 [ 7099 ] {08c0bb4f-a329-4e58-9ceb-f712c91b17d3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:01.302611 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:01.379205 [ 7099 ] {b230ecea-ed86-44ec-b096-e8b01548d4ee} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cramersVBiasCorrected(NULL, NULL, NULL);)
2024.11.04 23:20:01.384645 [ 7099 ] {b230ecea-ed86-44ec-b096-e8b01548d4ee} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:01.424733 [ 7099 ] {a8c279c5-0092-42b0-a052-fe7c7b3479bd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT curdate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT curdate(NULL, NULL, NULL);)
2024.11.04 23:20:01.427004 [ 7099 ] {a8c279c5-0092-42b0-a052-fe7c7b3479bd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT curdate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.464539 [ 7099 ] {b3a3516f-d88d-47ae-baa4-4dbf22ec2b39} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT currentDatabase(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT currentDatabase(NULL, NULL, NULL);)
2024.11.04 23:20:01.466701 [ 7099 ] {b3a3516f-d88d-47ae-baa4-4dbf22ec2b39} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT currentDatabase(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.475522 [ 3949 ] {94c50165-fcb8-4eb3-9046-20d1928b271f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50022) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:01.479557 [ 3949 ] {94c50165-fcb8-4eb3-9046-20d1928b271f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:01.635861 [ 7099 ] {6ef1da71-e419-4c13-b02c-d3023a5323c9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentProfiles doesn't match: passed 3, should be 0: In scope SELECT currentProfiles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT currentProfiles(NULL, NULL, NULL);)
2024.11.04 23:20:01.638532 [ 7099 ] {6ef1da71-e419-4c13-b02c-d3023a5323c9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentProfiles doesn't match: passed 3, should be 0: In scope SELECT currentProfiles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.678827 [ 7099 ] {843da7ec-41bb-49cd-a0fc-bed01a44cd84} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentRoles doesn't match: passed 3, should be 0: In scope SELECT currentRoles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT currentRoles(NULL, NULL, NULL);)
2024.11.04 23:20:01.681181 [ 7099 ] {843da7ec-41bb-49cd-a0fc-bed01a44cd84} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentRoles doesn't match: passed 3, should be 0: In scope SELECT currentRoles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.715247 [ 7099 ] {cb26eaf3-88ce-4eab-8ae6-007b1ef19092} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT currentSchemas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT currentSchemas(NULL, NULL, NULL);)
2024.11.04 23:20:01.717400 [ 7099 ] {cb26eaf3-88ce-4eab-8ae6-007b1ef19092} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT currentSchemas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.752062 [ 7099 ] {0ce1223a-2501-460d-b94f-72b39d1d8cd8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT currentUser(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT currentUser(NULL, NULL, NULL);)
2024.11.04 23:20:01.754584 [ 7099 ] {0ce1223a-2501-460d-b94f-72b39d1d8cd8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT currentUser(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.789719 [ 7099 ] {5d3c8e74-3b26-4095-85bf-46d7357b9573} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT current_database(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT current_database(NULL, NULL, NULL);)
2024.11.04 23:20:01.791775 [ 7099 ] {5d3c8e74-3b26-4095-85bf-46d7357b9573} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT current_database(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.827558 [ 7099 ] {35d3c085-91ea-47d6-9090-2a225cf4f1d6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT current_date(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT current_date(NULL, NULL, NULL);)
2024.11.04 23:20:01.830402 [ 7099 ] {35d3c085-91ea-47d6-9090-2a225cf4f1d6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT current_date(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:01.939010 [ 7099 ] {1cb6b8f3-f409-4cc9-8dfe-76b1c3010bcc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT current_schemas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT current_schemas(NULL, NULL, NULL);)
2024.11.04 23:20:01.959612 [ 7099 ] {1cb6b8f3-f409-4cc9-8dfe-76b1c3010bcc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT current_schemas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.004127 [ 7099 ] {ce9c78a1-d99a-48b4-9048-4023fc0a1db1} <Error> executeQuery: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT current_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT current_timestamp(NULL, NULL, NULL);)
2024.11.04 23:20:02.007520 [ 7099 ] {ce9c78a1-d99a-48b4-9048-4023fc0a1db1} <Error> TCPHandler: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT current_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.091668 [ 7099 ] {aca3a083-a52f-4f90-ad10-a504806051f3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT current_user(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT current_user(NULL, NULL, NULL);)
2024.11.04 23:20:02.093902 [ 7099 ] {aca3a083-a52f-4f90-ad10-a504806051f3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT current_user(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.133727 [ 7099 ] {5de9b9af-1783-4e3f-b54c-1a746a57bc79} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutFragment doesn't match: passed 3, should be 1: In scope SELECT cutFragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutFragment(NULL, NULL, NULL);)
2024.11.04 23:20:02.135884 [ 7099 ] {5de9b9af-1783-4e3f-b54c-1a746a57bc79} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutFragment doesn't match: passed 3, should be 1: In scope SELECT cutFragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.246834 [ 7099 ] {cc40b508-9081-473e-a9f4-cf8735ff89c6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutQueryString doesn't match: passed 3, should be 1: In scope SELECT cutQueryString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutQueryString(NULL, NULL, NULL);)
2024.11.04 23:20:02.248847 [ 7099 ] {cc40b508-9081-473e-a9f4-cf8735ff89c6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutQueryString doesn't match: passed 3, should be 1: In scope SELECT cutQueryString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.281895 [ 7099 ] {53b0826c-121d-4b0b-be13-a0808afe7fd4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutQueryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT cutQueryStringAndFragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutQueryStringAndFragment(NULL, NULL, NULL);)
2024.11.04 23:20:02.283763 [ 7099 ] {53b0826c-121d-4b0b-be13-a0808afe7fd4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutQueryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT cutQueryStringAndFragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.315962 [ 7099 ] {a64e45c7-55cf-4617-a8a9-a422fceb905f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomain(NULL, NULL, NULL);)
2024.11.04 23:20:02.317820 [ 7099 ] {a64e45c7-55cf-4617-a8a9-a422fceb905f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.349471 [ 7099 ] {20eac302-8795-4eca-8b9f-d4076aec3e36} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustom(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomainCustom(NULL, NULL, NULL);)
2024.11.04 23:20:02.351239 [ 7099 ] {20eac302-8795-4eca-8b9f-d4076aec3e36} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustom(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.383049 [ 7099 ] {2700bc79-cf68-409d-9d37-d3f3446c56a0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomainCustomRFC(NULL, NULL, NULL);)
2024.11.04 23:20:02.385090 [ 7099 ] {2700bc79-cf68-409d-9d37-d3f3446c56a0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.417499 [ 7099 ] {78f4c60e-00fd-4551-b698-72244f61d0b2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWW doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomainCustomWithWWW(NULL, NULL, NULL);)
2024.11.04 23:20:02.419329 [ 7099 ] {78f4c60e-00fd-4551-b698-72244f61d0b2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWW doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.452557 [ 7099 ] {36cc9722-42bc-4bc6-9c51-94402fe23223} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWWRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWWRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomainCustomWithWWWRFC(NULL, NULL, NULL);)
2024.11.04 23:20:02.454512 [ 7099 ] {36cc9722-42bc-4bc6-9c51-94402fe23223} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWWRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWWRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.491820 [ 3210 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.491885 [ 3197 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.491971 [ 3185 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.492031 [ 3192 ] {} <Warning> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.492139 [ 3182 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.492256 [ 3214 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.492519 [ 3194 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.493059 [ 3176 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:02.493290 [ 3487 ] {} <Error> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/table/test_13/03014_async_with_dedup_part_log/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:02.493895 [ 3502 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Operation timeout, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:02.495503 [ 7099 ] {0b37680f-19df-42f4-a235-0224c4efd1d0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomainRFC(NULL, NULL, NULL);)
2024.11.04 23:20:02.498571 [ 7099 ] {0b37680f-19df-42f4-a235-0224c4efd1d0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.534767 [ 3949 ] {9d2aa560-1985-479c-ab7f-36692f64b1a2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50034) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:02.536197 [ 7099 ] {421b06f6-f0d5-4d1e-90e6-ee291109e917} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWW doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomainWithWWW(NULL, NULL, NULL);)
2024.11.04 23:20:02.538369 [ 3949 ] {9d2aa560-1985-479c-ab7f-36692f64b1a2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:02.538756 [ 7099 ] {421b06f6-f0d5-4d1e-90e6-ee291109e917} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWW doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.574759 [ 7099 ] {a8594727-aa08-49c0-9988-be94fe184d40} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWWRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWWRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutToFirstSignificantSubdomainWithWWWRFC(NULL, NULL, NULL);)
2024.11.04 23:20:02.576792 [ 7099 ] {a8594727-aa08-49c0-9988-be94fe184d40} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWWRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWWRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.610579 [ 7099 ] {fdb9bf1b-2410-4688-aaf6-c207e19a0820} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutURLParameter doesn't match: passed 3, should be 2: In scope SELECT cutURLParameter(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutURLParameter(NULL, NULL, NULL);)
2024.11.04 23:20:02.612974 [ 7099 ] {fdb9bf1b-2410-4688-aaf6-c207e19a0820} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutURLParameter doesn't match: passed 3, should be 2: In scope SELECT cutURLParameter(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.648758 [ 7099 ] {5f92b08a-62ed-4352-87de-54d8fc078949} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function cutWWW doesn't match: passed 3, should be 1: In scope SELECT cutWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT cutWWW(NULL, NULL, NULL);)
2024.11.04 23:20:02.651013 [ 7099 ] {5f92b08a-62ed-4352-87de-54d8fc078949} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutWWW doesn't match: passed 3, should be 1: In scope SELECT cutWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.688575 [ 7099 ] {6474581e-0d1b-4bee-925a-5cbb2209d58d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function damerauLevenshteinDistance doesn't match: passed 3, should be 2: In scope SELECT damerauLevenshteinDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT damerauLevenshteinDistance(NULL, NULL, NULL);)
2024.11.04 23:20:02.690493 [ 7099 ] {6474581e-0d1b-4bee-925a-5cbb2209d58d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function damerauLevenshteinDistance doesn't match: passed 3, should be 2: In scope SELECT damerauLevenshteinDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.858913 [ 7099 ] {d7ad7c6d-2aa6-40df-8394-5b348e2ec3bd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function dateTime64ToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTime64ToSnowflake(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dateTime64ToSnowflake(NULL, NULL, NULL);)
2024.11.04 23:20:02.861551 [ 7099 ] {d7ad7c6d-2aa6-40df-8394-5b348e2ec3bd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function dateTime64ToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTime64ToSnowflake(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:02.899368 [ 7099 ] {7b46877c-641a-4dff-aeb6-a8d1ae15397e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function dateTimeToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTimeToSnowflake(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dateTimeToSnowflake(NULL, NULL, NULL);)
2024.11.04 23:20:02.901736 [ 7099 ] {7b46877c-641a-4dff-aeb6-a8d1ae15397e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function dateTimeToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTimeToSnowflake(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.029513 [ 7099 ] {7899ca91-7e2e-464b-9f28-f8481bdf5975} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeHTMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeHTMLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT decodeHTMLComponent(NULL, NULL, NULL);)
2024.11.04 23:20:03.031518 [ 7099 ] {7899ca91-7e2e-464b-9f28-f8481bdf5975} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeHTMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeHTMLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.063656 [ 7099 ] {e5b6cb52-efff-4b6e-90d2-e3bb01376080} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT decodeURLComponent(NULL, NULL, NULL);)
2024.11.04 23:20:03.065809 [ 7099 ] {e5b6cb52-efff-4b6e-90d2-e3bb01376080} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.098917 [ 7099 ] {d5162644-bfc7-461f-aa75-871af33046cd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLFormComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT decodeURLFormComponent(NULL, NULL, NULL);)
2024.11.04 23:20:03.101169 [ 7099 ] {d5162644-bfc7-461f-aa75-871af33046cd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLFormComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.133260 [ 7099 ] {e3b0e9b2-67c8-46cc-964d-22c50182a319} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeXMLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT decodeXMLComponent(NULL, NULL, NULL);)
2024.11.04 23:20:03.135250 [ 7099 ] {e3b0e9b2-67c8-46cc-964d-22c50182a319} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeXMLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.227759 [ 7099 ] {9068e009-bd9f-4844-8f8c-d64eb36cb85b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultProfiles doesn't match: passed 3, should be 0: In scope SELECT defaultProfiles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT defaultProfiles(NULL, NULL, NULL);)
2024.11.04 23:20:03.230885 [ 7099 ] {9068e009-bd9f-4844-8f8c-d64eb36cb85b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultProfiles doesn't match: passed 3, should be 0: In scope SELECT defaultProfiles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.290175 [ 7099 ] {ea98a660-22e2-41e6-aa03-d8f963918a99} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultRoles doesn't match: passed 3, should be 0: In scope SELECT defaultRoles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT defaultRoles(NULL, NULL, NULL);)
2024.11.04 23:20:03.292434 [ 7099 ] {ea98a660-22e2-41e6-aa03-d8f963918a99} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultRoles doesn't match: passed 3, should be 0: In scope SELECT defaultRoles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.331552 [ 7099 ] {54d10778-99c9-4baa-b25f-c385d6a1ee81} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultValueOfArgumentType doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfArgumentType(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT defaultValueOfArgumentType(NULL, NULL, NULL);)
2024.11.04 23:20:03.333663 [ 7099 ] {54d10778-99c9-4baa-b25f-c385d6a1ee81} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultValueOfArgumentType doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfArgumentType(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.446338 [ 7099 ] {ad5beed5-cbbe-4af2-89ae-1defa4fc5734} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultValueOfTypeName doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfTypeName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT defaultValueOfTypeName(NULL, NULL, NULL);)
2024.11.04 23:20:03.448902 [ 7099 ] {ad5beed5-cbbe-4af2-89ae-1defa4fc5734} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultValueOfTypeName doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfTypeName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.487968 [ 7099 ] {936f023f-1680-4b9e-aac5-d60eccc99029} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function degrees doesn't match: passed 3, should be 1: In scope SELECT degrees(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT degrees(NULL, NULL, NULL);)
2024.11.04 23:20:03.490172 [ 7099 ] {936f023f-1680-4b9e-aac5-d60eccc99029} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function degrees doesn't match: passed 3, should be 1: In scope SELECT degrees(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.530685 [ 7099 ] {50b03710-0def-4be0-b5cf-6e369a54eef6} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for aggregate function deltaSum. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT deltaSum(NULL, NULL, NULL);)
2024.11.04 23:20:03.532970 [ 7099 ] {50b03710-0def-4be0-b5cf-6e369a54eef6} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for aggregate function deltaSum. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.571580 [ 7099 ] {76ce7b00-3623-48a5-a9c2-2a96409bb10c} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function deltaSumTimestamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT deltaSumTimestamp(NULL, NULL, NULL);)
2024.11.04 23:20:03.573934 [ 7099 ] {76ce7b00-3623-48a5-a9c2-2a96409bb10c} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function deltaSumTimestamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.607880 [ 7099 ] {d8c5de53-0912-45a9-b26a-cc24bb635d3b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function demangle doesn't match: passed 3, should be 1: In scope SELECT demangle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT demangle(NULL, NULL, NULL);)
2024.11.04 23:20:03.610193 [ 7099 ] {d8c5de53-0912-45a9-b26a-cc24bb635d3b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function demangle doesn't match: passed 3, should be 1: In scope SELECT demangle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:03.711475 [ 7099 ] {578a9ee0-142a-4914-89d4-7b75c027983c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=32, jobs=32). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dense_rank(NULL, NULL, NULL);)
2024.11.04 23:20:03.715634 [ 7099 ] {578a9ee0-142a-4914-89d4-7b75c027983c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=32, jobs=32). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:03.815561 [ 7099 ] {4a7d903f-078f-46a1-88d7-471bc07e66cb} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectCharset' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT detectCharset(NULL, NULL, NULL);)
2024.11.04 23:20:03.818100 [ 7099 ] {4a7d903f-078f-46a1-88d7-471bc07e66cb} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectCharset' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:03.913398 [ 7099 ] {5a9898b2-14f8-4603-b1dc-1f74f7f90463} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT detectLanguage(NULL, NULL, NULL);)
2024.11.04 23:20:03.915393 [ 7099 ] {5a9898b2-14f8-4603-b1dc-1f74f7f90463} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:03.949533 [ 7099 ] {f0231af6-7106-43ad-949a-44899f636242} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectLanguageMixed' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT detectLanguageMixed(NULL, NULL, NULL);)
2024.11.04 23:20:03.951419 [ 7099 ] {f0231af6-7106-43ad-949a-44899f636242} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectLanguageMixed' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:03.984686 [ 7099 ] {ca2286a8-3899-472f-aa9c-b0246e3ff3b8} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectLanguageUnknown' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT detectLanguageUnknown(NULL, NULL, NULL);)
2024.11.04 23:20:03.986696 [ 7099 ] {ca2286a8-3899-472f-aa9c-b0246e3ff3b8} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectLanguageUnknown' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:04.020775 [ 7099 ] {0372185c-1a45-4268-aa54-9c412891e0f8} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectProgrammingLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT detectProgrammingLanguage(NULL, NULL, NULL);)
2024.11.04 23:20:04.022953 [ 7099 ] {0372185c-1a45-4268-aa54-9c412891e0f8} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectProgrammingLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:04.061753 [ 7099 ] {68791c76-9510-4331-ad8b-ee27de2786c3} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectTonality' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT detectTonality(NULL, NULL, NULL);)
2024.11.04 23:20:04.063637 [ 7099 ] {68791c76-9510-4331-ad8b-ee27de2786c3} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectTonality' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:04.179238 [ 7099 ] {6f059e57-dffa-49af-9678-1fa8d79294be} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGet(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dictGet(NULL, NULL, NULL);)
2024.11.04 23:20:04.181530 [ 7099 ] {6f059e57-dffa-49af-9678-1fa8d79294be} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGet(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:04.221523 [ 7099 ] {7c1856e4-d5ac-4d5d-8dda-6f66f4054367} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGetAll, expected a const string.: In scope SELECT dictGetAll(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dictGetAll(NULL, NULL, NULL);)
2024.11.04 23:20:04.223840 [ 7099 ] {7c1856e4-d5ac-4d5d-8dda-6f66f4054367} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGetAll, expected a const string.: In scope SELECT dictGetAll(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:04.262403 [ 7099 ] {12aea03d-57f5-4ff8-bb37-2d08672ba8b8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function dictGetChildren doesn't match: passed 3, should be 2: In scope SELECT dictGetChildren(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dictGetChildren(NULL, NULL, NULL);)
2024.11.04 23:20:04.264793 [ 7099 ] {12aea03d-57f5-4ff8-bb37-2d08672ba8b8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function dictGetChildren doesn't match: passed 3, should be 2: In scope SELECT dictGetChildren(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:04.659363 [ 7099 ] {03d12410-39ed-4d97-9040-88759d11670b} <Error> executeQuery: Code: 1. DB::Exception: Expected const String column: In scope SELECT dictGetDescendants(NULL, NULL, NULL). (UNSUPPORTED_METHOD) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dictGetDescendants(NULL, NULL, NULL);)
2024.11.04 23:20:04.661793 [ 7099 ] {03d12410-39ed-4d97-9040-88759d11670b} <Error> TCPHandler: Code: 1. DB::Exception: Expected const String column: In scope SELECT dictGetDescendants(NULL, NULL, NULL). (UNSUPPORTED_METHOD)
2024.11.04 23:20:04.883748 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=76, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:04.898395 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:05.103205 [ 7099 ] {c58cdf36-7e40-423a-af99-0d2a7939d8ee} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function dictGetHierarchy doesn't match: passed 3, should be 2: In scope SELECT dictGetHierarchy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dictGetHierarchy(NULL, NULL, NULL);)
2024.11.04 23:20:05.105758 [ 7099 ] {c58cdf36-7e40-423a-af99-0d2a7939d8ee} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function dictGetHierarchy doesn't match: passed 3, should be 2: In scope SELECT dictGetHierarchy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:05.358733 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:05.358874 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:05.464743 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/gnq/hfjzqvslpcxgakofxdsqahgpgyrog, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 20de2286-93fb-4243-ac20-4e8029309be7, upload has finished false.
2024.11.04 23:20:05.464940 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/gnq/hfjzqvslpcxgakofxdsqahgpgyrog, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 20de2286-93fb-4243-ac20-4e8029309be7, upload has finished false
2024.11.04 23:20:05.506847 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/gnq/hfjzqvslpcxgakofxdsqahgpgyrog, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 20de2286-93fb-4243-ac20-4e8029309be7, upload has finished false
2024.11.04 23:20:05.511609 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:05.841499 [ 7099 ] {74322458-aa30-4cc4-ab3e-04385fc39853} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetOrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dictGetOrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:05.843987 [ 7099 ] {74322458-aa30-4cc4-ab3e-04385fc39853} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetOrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:05.878987 [ 7099 ] {06dd3aa8-4af7-49ed-9ebe-e371ba25d7fc} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetOrNull(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dictGetOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:05.881394 [ 7099 ] {06dd3aa8-4af7-49ed-9ebe-e371ba25d7fc} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetOrNull(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:06.030389 [ 3949 ] {0351e9c5-b462-42a4-826b-a230d0570b61} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39878) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:06.033661 [ 3949 ] {0351e9c5-b462-42a4-826b-a230d0570b61} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:06.088927 [ 3413 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.089023 [ 3318 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.089127 [ 3345 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.089199 [ 3332 ] {} <Warning> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.089376 [ 3198 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.089470 [ 3431 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.089594 [ 3404 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.089747 [ 3394 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:06.090302 [ 3390 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:06.090578 [ 3438 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired (fault injected on send). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1605: Coordination::ZooKeeper::maybeInjectSendFault() @ 0x000000003c15beaf
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b012
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
9. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
10. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
13. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
14. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
15. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
16. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
18. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
19. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:06.090719 [ 3337 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:06.097079 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:06.946651 [ 7099 ] {f193a83e-9a7c-46e0-b5fd-eafb2b14bcb3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function displayName doesn't match: passed 3, should be 0: In scope SELECT displayName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT displayName(NULL, NULL, NULL);)
2024.11.04 23:20:06.948725 [ 7099 ] {f193a83e-9a7c-46e0-b5fd-eafb2b14bcb3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function displayName doesn't match: passed 3, should be 0: In scope SELECT displayName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:06.982237 [ 7099 ] {303fa6d1-6e1a-4e6f-b4e7-76c27116ee10} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT distanceL1(NULL, NULL, NULL);)
2024.11.04 23:20:06.984698 [ 7099 ] {303fa6d1-6e1a-4e6f-b4e7-76c27116ee10} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.035859 [ 7099 ] {39b2c91d-dfdf-455b-b606-97747e22ac5d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT distanceL2(NULL, NULL, NULL);)
2024.11.04 23:20:07.037876 [ 7099 ] {39b2c91d-dfdf-455b-b606-97747e22ac5d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.073965 [ 7099 ] {636adb0c-8587-4270-814e-109e28437b2e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT distanceL2Squared(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT distanceL2Squared(NULL, NULL, NULL);)
2024.11.04 23:20:07.075880 [ 7099 ] {636adb0c-8587-4270-814e-109e28437b2e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT distanceL2Squared(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.116962 [ 3582 ] {} <Warning> DNSResolver: Cannot resolve host (localhosx), error 0: Host not found.
2024.11.04 23:20:07.134598 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 11 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:07.139893 [ 7099 ] {929f4b15-2ae4-4565-873e-6d3922f6d759} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT distanceLinf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT distanceLinf(NULL, NULL, NULL);)
2024.11.04 23:20:07.143092 [ 7099 ] {929f4b15-2ae4-4565-873e-6d3922f6d759} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT distanceLinf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.208115 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:07.222398 [ 7099 ] {2f32dcc4-4009-41f0-9e35-b5f6a353646d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function divide doesn't match: passed 3, should be 2: In scope SELECT divide(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT divide(NULL, NULL, NULL);)
2024.11.04 23:20:07.224193 [ 7099 ] {2f32dcc4-4009-41f0-9e35-b5f6a353646d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function divide doesn't match: passed 3, should be 2: In scope SELECT divide(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.377048 [ 7099 ] {1cf97a7b-4c74-4839-85cd-b5c2b7bebe2e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function domain doesn't match: passed 3, should be 1: In scope SELECT domain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT domain(NULL, NULL, NULL);)
2024.11.04 23:20:07.381139 [ 7099 ] {1cf97a7b-4c74-4839-85cd-b5c2b7bebe2e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function domain doesn't match: passed 3, should be 1: In scope SELECT domain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.465461 [ 7099 ] {35e1d7a2-8a98-46e3-a790-ec0a3d2ae617} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function domainRFC doesn't match: passed 3, should be 1: In scope SELECT domainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT domainRFC(NULL, NULL, NULL);)
2024.11.04 23:20:07.468695 [ 7099 ] {35e1d7a2-8a98-46e3-a790-ec0a3d2ae617} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function domainRFC doesn't match: passed 3, should be 1: In scope SELECT domainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.685845 [ 7099 ] {9f9b02c6-39be-47f9-a88c-552b2680d58c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWW doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT domainWithoutWWW(NULL, NULL, NULL);)
2024.11.04 23:20:07.687896 [ 7099 ] {9f9b02c6-39be-47f9-a88c-552b2680d58c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWW doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWW(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.777116 [ 7099 ] {1269dc52-520b-4a37-a0ee-f159a03e0e18} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWWRFC doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWWRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT domainWithoutWWWRFC(NULL, NULL, NULL);)
2024.11.04 23:20:07.780050 [ 7099 ] {1269dc52-520b-4a37-a0ee-f159a03e0e18} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWWRFC doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWWRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.905419 [ 7099 ] {e8aeb42b-6678-419d-8d15-df0ee9fb9e14} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT dotProduct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dotProduct(NULL, NULL, NULL);)
2024.11.04 23:20:07.907988 [ 7099 ] {e8aeb42b-6678-419d-8d15-df0ee9fb9e14} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT dotProduct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.951544 [ 7099 ] {27010e2e-bbe6-40aa-94aa-ccb090815859} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function dumpColumnStructure doesn't match: passed 3, should be 1: In scope SELECT dumpColumnStructure(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT dumpColumnStructure(NULL, NULL, NULL);)
2024.11.04 23:20:07.954874 [ 7099 ] {27010e2e-bbe6-40aa-94aa-ccb090815859} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function dumpColumnStructure doesn't match: passed 3, should be 1: In scope SELECT dumpColumnStructure(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:07.991699 [ 7099 ] {5a47dcdc-653a-4de0-8062-44d95d9f6193} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function e doesn't match: passed 3, should be 0: In scope SELECT e(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT e(NULL, NULL, NULL);)
2024.11.04 23:20:07.993963 [ 7099 ] {5a47dcdc-653a-4de0-8062-44d95d9f6193} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function e doesn't match: passed 3, should be 0: In scope SELECT e(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.027039 [ 7099 ] {2a5a8e53-4636-4319-847b-d2e113cb00fe} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT editDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT editDistance(NULL, NULL, NULL);)
2024.11.04 23:20:08.027619 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:08.029743 [ 7099 ] {2a5a8e53-4636-4319-847b-d2e113cb00fe} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT editDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.061434 [ 7099 ] {71bcb1ce-2bad-4e75-ab20-16010f902fd6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function empty doesn't match: passed 3, should be 1: In scope SELECT empty(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT empty(NULL, NULL, NULL);)
2024.11.04 23:20:08.063481 [ 7099 ] {71bcb1ce-2bad-4e75-ab20-16010f902fd6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function empty doesn't match: passed 3, should be 1: In scope SELECT empty(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.095540 [ 7099 ] {93d32a66-c8f1-4917-b93e-d0e38db45f35} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayDate doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayDate(NULL, NULL, NULL);)
2024.11.04 23:20:08.097747 [ 7099 ] {93d32a66-c8f1-4917-b93e-d0e38db45f35} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayDate doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.130103 [ 7099 ] {0dcd8dea-d8f0-4f55-a6dd-3055174501d5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayDateTime doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDateTime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayDateTime(NULL, NULL, NULL);)
2024.11.04 23:20:08.132083 [ 7099 ] {0dcd8dea-d8f0-4f55-a6dd-3055174501d5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayDateTime doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDateTime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.164431 [ 7099 ] {f7230f9d-b916-43b3-94d2-416b264ad789} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayFloat32(NULL, NULL, NULL);)
2024.11.04 23:20:08.166927 [ 7099 ] {f7230f9d-b916-43b3-94d2-416b264ad789} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.288183 [ 7099 ] {6ef651ec-4c75-49d5-8789-fc541550c7b8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayFloat64(NULL, NULL, NULL);)
2024.11.04 23:20:08.290209 [ 7099 ] {6ef651ec-4c75-49d5-8789-fc541550c7b8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.328522 [ 7099 ] {6b4d0cc1-84ad-4148-8be0-da1a51e217f9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayInt16(NULL, NULL, NULL);)
2024.11.04 23:20:08.330772 [ 7099 ] {6b4d0cc1-84ad-4148-8be0-da1a51e217f9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.371320 [ 7099 ] {bc924bd5-b68d-4e50-a436-6c9bf9e67011} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayInt32(NULL, NULL, NULL);)
2024.11.04 23:20:08.373707 [ 7099 ] {bc924bd5-b68d-4e50-a436-6c9bf9e67011} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.407836 [ 7099 ] {b81465fd-0fe1-482e-a734-247bb5b4adcc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayInt64(NULL, NULL, NULL);)
2024.11.04 23:20:08.410128 [ 7099 ] {b81465fd-0fe1-482e-a734-247bb5b4adcc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.444791 [ 7099 ] {6ba6356b-c489-4605-97e8-a87767a84104} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayInt8(NULL, NULL, NULL);)
2024.11.04 23:20:08.447167 [ 7099 ] {6ba6356b-c489-4605-97e8-a87767a84104} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.480008 [ 7099 ] {1132c93c-6976-409a-b6c8-502d7a5949e0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayString doesn't match: passed 3, should be 0: In scope SELECT emptyArrayString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayString(NULL, NULL, NULL);)
2024.11.04 23:20:08.482481 [ 7099 ] {1132c93c-6976-409a-b6c8-502d7a5949e0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayString doesn't match: passed 3, should be 0: In scope SELECT emptyArrayString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.514038 [ 7099 ] {ed6826d2-49cb-4a99-99b3-da9228b31fd4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayToSingle doesn't match: passed 3, should be 1: In scope SELECT emptyArrayToSingle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayToSingle(NULL, NULL, NULL);)
2024.11.04 23:20:08.516155 [ 7099 ] {ed6826d2-49cb-4a99-99b3-da9228b31fd4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayToSingle doesn't match: passed 3, should be 1: In scope SELECT emptyArrayToSingle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.547826 [ 7099 ] {03dae361-d718-49ab-aa9f-9702aef1c260} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayUInt16(NULL, NULL, NULL);)
2024.11.04 23:20:08.550004 [ 7099 ] {03dae361-d718-49ab-aa9f-9702aef1c260} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.580686 [ 7099 ] {a588673b-0747-4597-9687-f1b39e94a3f3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayUInt32(NULL, NULL, NULL);)
2024.11.04 23:20:08.582511 [ 7099 ] {a588673b-0747-4597-9687-f1b39e94a3f3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.611869 [ 7099 ] {155a170c-f7d5-4273-adc9-1ee52bf31964} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayUInt64(NULL, NULL, NULL);)
2024.11.04 23:20:08.613509 [ 7099 ] {155a170c-f7d5-4273-adc9-1ee52bf31964} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.641770 [ 7099 ] {9c0b971b-8b65-4c7d-8c05-544b0b95340c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT emptyArrayUInt8(NULL, NULL, NULL);)
2024.11.04 23:20:08.643324 [ 7099 ] {9c0b971b-8b65-4c7d-8c05-544b0b95340c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.674259 [ 7099 ] {942f2038-2170-4d29-98e5-fce40636ecdd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function enabledProfiles doesn't match: passed 3, should be 0: In scope SELECT enabledProfiles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT enabledProfiles(NULL, NULL, NULL);)
2024.11.04 23:20:08.676561 [ 7099 ] {942f2038-2170-4d29-98e5-fce40636ecdd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function enabledProfiles doesn't match: passed 3, should be 0: In scope SELECT enabledProfiles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.705373 [ 7099 ] {e3783b5e-dfbb-4f4e-96a2-ba1d4eac541e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function enabledRoles doesn't match: passed 3, should be 0: In scope SELECT enabledRoles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT enabledRoles(NULL, NULL, NULL);)
2024.11.04 23:20:08.707065 [ 7099 ] {e3783b5e-dfbb-4f4e-96a2-ba1d4eac541e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function enabledRoles doesn't match: passed 3, should be 0: In scope SELECT enabledRoles(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.736330 [ 7099 ] {d0eed339-f792-4bbc-a4b9-28bc1a845018} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function encodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT encodeURLComponent(NULL, NULL, NULL);)
2024.11.04 23:20:08.737868 [ 7099 ] {d0eed339-f792-4bbc-a4b9-28bc1a845018} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function encodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.799729 [ 7099 ] {ade18c91-f880-43c5-8c9d-21f8845af9f0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function encodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLFormComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT encodeURLFormComponent(NULL, NULL, NULL);)
2024.11.04 23:20:08.801936 [ 7099 ] {ade18c91-f880-43c5-8c9d-21f8845af9f0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function encodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLFormComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:08.834434 [ 7099 ] {44f2c323-486b-426b-a0ab-836028bee337} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function encodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeXMLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT encodeXMLComponent(NULL, NULL, NULL);)
2024.11.04 23:20:08.863369 [ 7099 ] {44f2c323-486b-426b-a0ab-836028bee337} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function encodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeXMLComponent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.059807 [ 7099 ] {0493fe9a-5b7f-49d3-8025-a4cbd3e1c719} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function endsWith doesn't match: passed 3, should be 2: In scope SELECT endsWith(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT endsWith(NULL, NULL, NULL);)
2024.11.04 23:20:09.062337 [ 7099 ] {0493fe9a-5b7f-49d3-8025-a4cbd3e1c719} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function endsWith doesn't match: passed 3, should be 2: In scope SELECT endsWith(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.100022 [ 7099 ] {3919013d-832c-4088-9f88-df3b9855f59c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function endsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT endsWithUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT endsWithUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:09.200672 [ 7099 ] {3919013d-832c-4088-9f88-df3b9855f59c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function endsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT endsWithUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.322291 [ 3949 ] {2f67858d-81a3-43dd-9d7e-1e2110bbebc6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:39904) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:09.325545 [ 3949 ] {2f67858d-81a3-43dd-9d7e-1e2110bbebc6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:09.352865 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 5 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:09.360829 [ 7099 ] {5e7f8c98-9dee-4f7b-a296-53838f0d2183} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function equals doesn't match: passed 3, should be 2: In scope SELECT equals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT equals(NULL, NULL, NULL);)
2024.11.04 23:20:09.362914 [ 7099 ] {5e7f8c98-9dee-4f7b-a296-53838f0d2183} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function equals doesn't match: passed 3, should be 2: In scope SELECT equals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.396093 [ 7099 ] {8c357030-7ad9-4e8d-a2b0-9c6e8976ab76} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function erf doesn't match: passed 3, should be 1: In scope SELECT erf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT erf(NULL, NULL, NULL);)
2024.11.04 23:20:09.465668 [ 7099 ] {8c357030-7ad9-4e8d-a2b0-9c6e8976ab76} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function erf doesn't match: passed 3, should be 1: In scope SELECT erf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.509382 [ 7099 ] {2902d195-c5e8-4f67-8d45-f0d6e44c3d0d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function erfc doesn't match: passed 3, should be 1: In scope SELECT erfc(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT erfc(NULL, NULL, NULL);)
2024.11.04 23:20:09.512357 [ 7099 ] {2902d195-c5e8-4f67-8d45-f0d6e44c3d0d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function erfc doesn't match: passed 3, should be 1: In scope SELECT erfc(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.547304 [ 7099 ] {a560996b-2ef4-4e8c-a995-682db04c5f25} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function errorCodeToName doesn't match: passed 3, should be 1: In scope SELECT errorCodeToName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT errorCodeToName(NULL, NULL, NULL);)
2024.11.04 23:20:09.549797 [ 7099 ] {a560996b-2ef4-4e8c-a995-682db04c5f25} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function errorCodeToName doesn't match: passed 3, should be 1: In scope SELECT errorCodeToName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.739883 [ 7099 ] {08415ee9-767d-484b-96d8-79f62f082da6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function exp doesn't match: passed 3, should be 1: In scope SELECT exp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT exp(NULL, NULL, NULL);)
2024.11.04 23:20:09.742302 [ 7099 ] {08415ee9-767d-484b-96d8-79f62f082da6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function exp doesn't match: passed 3, should be 1: In scope SELECT exp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.777651 [ 7099 ] {2479fa5e-e2f4-4ad6-866d-b7448739799d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function exp10 doesn't match: passed 3, should be 1: In scope SELECT exp10(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT exp10(NULL, NULL, NULL);)
2024.11.04 23:20:09.779751 [ 7099 ] {2479fa5e-e2f4-4ad6-866d-b7448739799d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function exp10 doesn't match: passed 3, should be 1: In scope SELECT exp10(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:09.812660 [ 7099 ] {ba65252d-60d3-4261-bc5c-49e5309c6cdb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function exp2 doesn't match: passed 3, should be 1: In scope SELECT exp2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT exp2(NULL, NULL, NULL);)
2024.11.04 23:20:09.815045 [ 7099 ] {ba65252d-60d3-4261-bc5c-49e5309c6cdb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function exp2 doesn't match: passed 3, should be 1: In scope SELECT exp2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.015917 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:10.141380 [ 7099 ] {227cf66f-1aa2-4fcf-8aba-6b2012f072e7} <Error> executeQuery: Code: 36. DB::Exception: Function exponentialTimeDecayedAvg takes exactly one parameter. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT exponentialTimeDecayedAvg(NULL, NULL, NULL);)
2024.11.04 23:20:10.143614 [ 7099 ] {227cf66f-1aa2-4fcf-8aba-6b2012f072e7} <Error> TCPHandler: Code: 36. DB::Exception: Function exponentialTimeDecayedAvg takes exactly one parameter. (BAD_ARGUMENTS)
2024.11.04 23:20:10.175077 [ 7099 ] {0552fab4-16dd-4ae4-b379-ee60d1871005} <Error> executeQuery: Code: 36. DB::Exception: Function exponentialTimeDecayedCount takes exactly one parameter. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT exponentialTimeDecayedCount(NULL, NULL, NULL);)
2024.11.04 23:20:10.177006 [ 7099 ] {0552fab4-16dd-4ae4-b379-ee60d1871005} <Error> TCPHandler: Code: 36. DB::Exception: Function exponentialTimeDecayedCount takes exactly one parameter. (BAD_ARGUMENTS)
2024.11.04 23:20:10.207921 [ 7099 ] {2ec923ed-7057-46d5-8ed6-7d1e5dc5783c} <Error> executeQuery: Code: 36. DB::Exception: Function exponentialTimeDecayedMax takes exactly one parameter. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT exponentialTimeDecayedMax(NULL, NULL, NULL);)
2024.11.04 23:20:10.209870 [ 7099 ] {2ec923ed-7057-46d5-8ed6-7d1e5dc5783c} <Error> TCPHandler: Code: 36. DB::Exception: Function exponentialTimeDecayedMax takes exactly one parameter. (BAD_ARGUMENTS)
2024.11.04 23:20:10.241707 [ 7099 ] {3a3e957a-24fa-4014-bdc6-5a44549cb421} <Error> executeQuery: Code: 36. DB::Exception: Function exponentialTimeDecayedSum takes exactly one parameter. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT exponentialTimeDecayedSum(NULL, NULL, NULL);)
2024.11.04 23:20:10.243535 [ 7099 ] {3a3e957a-24fa-4014-bdc6-5a44549cb421} <Error> TCPHandler: Code: 36. DB::Exception: Function exponentialTimeDecayedSum takes exactly one parameter. (BAD_ARGUMENTS)
2024.11.04 23:20:10.273672 [ 7099 ] {0005b54c-0a70-4b6a-8d84-374abfcf18c7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extract doesn't match: passed 3, should be 2: In scope SELECT extract(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extract(NULL, NULL, NULL);)
2024.11.04 23:20:10.275658 [ 7099 ] {0005b54c-0a70-4b6a-8d84-374abfcf18c7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extract doesn't match: passed 3, should be 2: In scope SELECT extract(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.312697 [ 7099 ] {44942848-d42d-4eb7-a559-6b5cd0959002} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAll doesn't match: passed 3, should be 2: In scope SELECT extractAll(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractAll(NULL, NULL, NULL);)
2024.11.04 23:20:10.314564 [ 7099 ] {44942848-d42d-4eb7-a559-6b5cd0959002} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAll doesn't match: passed 3, should be 2: In scope SELECT extractAll(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.395948 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_105_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=76, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:10.414523 [ 7099 ] {d04c0ddd-1ff4-4f1f-9185-7daea72d788d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroups(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractAllGroups(NULL, NULL, NULL);)
2024.11.04 23:20:10.416593 [ 7099 ] {d04c0ddd-1ff4-4f1f-9185-7daea72d788d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroups(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.452847 [ 7099 ] {5c87cf6e-1479-4215-b7c0-1be0b2e471c2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsHorizontal(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractAllGroupsHorizontal(NULL, NULL, NULL);)
2024.11.04 23:20:10.455183 [ 7099 ] {5c87cf6e-1479-4215-b7c0-1be0b2e471c2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsHorizontal(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.493479 [ 7099 ] {2b47c4c0-348a-4146-8bd0-66d0d85a78a1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsVertical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractAllGroupsVertical(NULL, NULL, NULL);)
2024.11.04 23:20:10.495653 [ 7099 ] {2b47c4c0-348a-4146-8bd0-66d0d85a78a1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsVertical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.528414 [ 7099 ] {7f8ef39f-68ef-40dc-a3ac-a26726a2cec7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractGroups doesn't match: passed 3, should be 2: In scope SELECT extractGroups(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractGroups(NULL, NULL, NULL);)
2024.11.04 23:20:10.530591 [ 7099 ] {7f8ef39f-68ef-40dc-a3ac-a26726a2cec7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractGroups doesn't match: passed 3, should be 2: In scope SELECT extractGroups(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.790356 [ 7099 ] {5d190fc9-01bd-4180-8d2f-547cd56ecccc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractTextFromHTML doesn't match: passed 3, should be 1: In scope SELECT extractTextFromHTML(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractTextFromHTML(NULL, NULL, NULL);)
2024.11.04 23:20:10.792354 [ 7099 ] {5d190fc9-01bd-4180-8d2f-547cd56ecccc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractTextFromHTML doesn't match: passed 3, should be 1: In scope SELECT extractTextFromHTML(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.827265 [ 7099 ] {18eabc49-6b12-4bc5-8f9d-78ca64398567} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractURLParameter doesn't match: passed 3, should be 2: In scope SELECT extractURLParameter(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractURLParameter(NULL, NULL, NULL);)
2024.11.04 23:20:10.829619 [ 7099 ] {18eabc49-6b12-4bc5-8f9d-78ca64398567} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractURLParameter doesn't match: passed 3, should be 2: In scope SELECT extractURLParameter(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.866998 [ 7099 ] {6c925a0e-e936-4728-8881-fc8cd8ce18d1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractURLParameterNames doesn't match: passed 3, should be 1: In scope SELECT extractURLParameterNames(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractURLParameterNames(NULL, NULL, NULL);)
2024.11.04 23:20:10.869418 [ 7099 ] {6c925a0e-e936-4728-8881-fc8cd8ce18d1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractURLParameterNames doesn't match: passed 3, should be 1: In scope SELECT extractURLParameterNames(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.906623 [ 7099 ] {b214164c-3939-4c4e-a391-82f37f766f47} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function extractURLParameters doesn't match: passed 3, should be 1: In scope SELECT extractURLParameters(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT extractURLParameters(NULL, NULL, NULL);)
2024.11.04 23:20:10.910854 [ 7099 ] {b214164c-3939-4c4e-a391-82f37f766f47} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractURLParameters doesn't match: passed 3, should be 1: In scope SELECT extractURLParameters(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:10.947042 [ 7099 ] {b1512f43-b05e-4414-96c3-4a52426a5b1b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function factorial doesn't match: passed 3, should be 1: In scope SELECT factorial(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT factorial(NULL, NULL, NULL);)
2024.11.04 23:20:10.949600 [ 7099 ] {b1512f43-b05e-4414-96c3-4a52426a5b1b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function factorial doesn't match: passed 3, should be 1: In scope SELECT factorial(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.163985 [ 7099 ] {d199dbcc-c88d-49a3-9ebf-4e05a2702fb0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function file doesn't match: passed 3, should be 1 or 2: In scope SELECT file(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT file(NULL, NULL, NULL);)
2024.11.04 23:20:11.166017 [ 7099 ] {d199dbcc-c88d-49a3-9ebf-4e05a2702fb0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function file doesn't match: passed 3, should be 1 or 2: In scope SELECT file(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.417753 [ 7099 ] {80424a60-e349-4dc4-b206-0ce5001a8496} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function finalizeAggregation doesn't match: passed 3, should be 1: In scope SELECT finalizeAggregation(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT finalizeAggregation(NULL, NULL, NULL);)
2024.11.04 23:20:11.420377 [ 7099 ] {80424a60-e349-4dc4-b206-0ce5001a8496} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function finalizeAggregation doesn't match: passed 3, should be 1: In scope SELECT finalizeAggregation(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.452115 [ 7099 ] {d8ebb7fc-fbb8-46a9-8ded-b9cdb4a9feb5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function firstLine doesn't match: passed 3, should be 1: In scope SELECT firstLine(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT firstLine(NULL, NULL, NULL);)
2024.11.04 23:20:11.454517 [ 7099 ] {d8ebb7fc-fbb8-46a9-8ded-b9cdb4a9feb5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstLine doesn't match: passed 3, should be 1: In scope SELECT firstLine(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.486364 [ 7099 ] {21b7892c-efbd-4d43-9fe2-9c8dbfabe195} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT firstSignificantSubdomain(NULL, NULL, NULL);)
2024.11.04 23:20:11.488679 [ 7099 ] {21b7892c-efbd-4d43-9fe2-9c8dbfabe195} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.522399 [ 7099 ] {bfa9308a-f7d2-484d-ac06-0e97ef39e4e9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustom(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT firstSignificantSubdomainCustom(NULL, NULL, NULL);)
2024.11.04 23:20:11.524434 [ 7099 ] {bfa9308a-f7d2-484d-ac06-0e97ef39e4e9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustom(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.557620 [ 7099 ] {4c9052df-ff45-4ff2-9d82-d0e2244598ad} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustomRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT firstSignificantSubdomainCustomRFC(NULL, NULL, NULL);)
2024.11.04 23:20:11.559813 [ 7099 ] {4c9052df-ff45-4ff2-9d82-d0e2244598ad} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustomRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.614297 [ 7099 ] {877a38e8-4326-4d8f-b963-0fbca07b2f35} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT firstSignificantSubdomainRFC(NULL, NULL, NULL);)
2024.11.04 23:20:11.616496 [ 7099 ] {877a38e8-4326-4d8f-b963-0fbca07b2f35} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.712583 [ 7099 ] {78897caa-9f89-466f-8b52-d99792802aeb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT first_value(NULL, NULL, NULL);)
2024.11.04 23:20:11.718239 [ 7099 ] {78897caa-9f89-466f-8b52-d99792802aeb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:11.734997 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 31 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:11.750220 [ 7099 ] {86e85c26-bef7-4b3f-b9c5-12ceaa158b74} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT first_value_respect_nulls(NULL, NULL, NULL);)
2024.11.04 23:20:11.752313 [ 7099 ] {86e85c26-bef7-4b3f-b9c5-12ceaa158b74} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.784958 [ 7099 ] {2014d6bd-5062-4412-999f-3f4ee905262e} <Error> executeQuery: Code: 43. DB::Exception: First argument (trace) for function flameGraph must be Array(UInt64), but it has type Nullable(Nothing). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT flameGraph(NULL, NULL, NULL);)
2024.11.04 23:20:11.786615 [ 7099 ] {2014d6bd-5062-4412-999f-3f4ee905262e} <Error> TCPHandler: Code: 43. DB::Exception: First argument (trace) for function flameGraph must be Array(UInt64), but it has type Nullable(Nothing). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:11.942834 [ 7099 ] {aeee479f-9748-4190-aa09-4a16ccda2cf6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT flatten(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT flatten(NULL, NULL, NULL);)
2024.11.04 23:20:11.945184 [ 7099 ] {aeee479f-9748-4190-aa09-4a16ccda2cf6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT flatten(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:11.981864 [ 7099 ] {e27faad5-396d-404c-a30a-cfc3112ee08a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function flattenTuple doesn't match: passed 3, should be 1: In scope SELECT flattenTuple(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT flattenTuple(NULL, NULL, NULL);)
2024.11.04 23:20:12.078180 [ 7099 ] {e27faad5-396d-404c-a30a-cfc3112ee08a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function flattenTuple doesn't match: passed 3, should be 1: In scope SELECT flattenTuple(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.109952 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:12.329490 [ 7099 ] {e57427aa-0c5b-4a2c-b79e-1eee7233f5a1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQuery doesn't match: passed 3, should be 1: In scope SELECT formatQuery(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatQuery(NULL, NULL, NULL);)
2024.11.04 23:20:12.331796 [ 7099 ] {e57427aa-0c5b-4a2c-b79e-1eee7233f5a1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQuery doesn't match: passed 3, should be 1: In scope SELECT formatQuery(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.365111 [ 7099 ] {4862d371-8980-47b7-aa6b-530a91e77633} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQueryOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQueryOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatQueryOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:12.367120 [ 7099 ] {4862d371-8980-47b7-aa6b-530a91e77633} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQueryOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQueryOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.400194 [ 7099 ] {340fada4-9499-423f-bf6f-2261c4bda16a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLine doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLine(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatQuerySingleLine(NULL, NULL, NULL);)
2024.11.04 23:20:12.402330 [ 7099 ] {340fada4-9499-423f-bf6f-2261c4bda16a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLine doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLine(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.435812 [ 7099 ] {105636bc-c0ad-4ac6-9a59-4c9df6c1fb75} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLineOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLineOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatQuerySingleLineOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:12.438418 [ 7099 ] {105636bc-c0ad-4ac6-9a59-4c9df6c1fb75} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLineOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLineOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.470529 [ 7099 ] {5fb9c77a-293f-4d97-a4d3-d1ef06136762} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableDecimalSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableDecimalSize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatReadableDecimalSize(NULL, NULL, NULL);)
2024.11.04 23:20:12.472818 [ 7099 ] {5fb9c77a-293f-4d97-a4d3-d1ef06136762} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableDecimalSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableDecimalSize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.507318 [ 7099 ] {396d4702-b977-4121-8f5e-70cec651e03d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableQuantity doesn't match: passed 3, should be 1: In scope SELECT formatReadableQuantity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatReadableQuantity(NULL, NULL, NULL);)
2024.11.04 23:20:12.510234 [ 7099 ] {396d4702-b977-4121-8f5e-70cec651e03d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableQuantity doesn't match: passed 3, should be 1: In scope SELECT formatReadableQuantity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.546947 [ 7099 ] {6d76266d-6108-4c97-a503-1c2d79caedce} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableSize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatReadableSize(NULL, NULL, NULL);)
2024.11.04 23:20:12.548923 [ 7099 ] {6d76266d-6108-4c97-a503-1c2d79caedce} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableSize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.581218 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_101_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:12.633758 [ 7099 ] {83452dc2-a83b-4d5e-b12d-e3cff496dca8} <Error> executeQuery: Code: 43. DB::Exception: First argument to formatRow must be a format name: In scope SELECT formatRow(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatRow(NULL, NULL, NULL);)
2024.11.04 23:20:12.635753 [ 7099 ] {83452dc2-a83b-4d5e-b12d-e3cff496dca8} <Error> TCPHandler: Code: 43. DB::Exception: First argument to formatRow must be a format name: In scope SELECT formatRow(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:12.665908 [ 7099 ] {591c79eb-4391-41fa-abf6-ac428172ac53} <Error> executeQuery: Code: 43. DB::Exception: First argument to formatRowNoNewline must be a format name: In scope SELECT formatRowNoNewline(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT formatRowNoNewline(NULL, NULL, NULL);)
2024.11.04 23:20:12.667913 [ 7099 ] {591c79eb-4391-41fa-abf6-ac428172ac53} <Error> TCPHandler: Code: 43. DB::Exception: First argument to formatRowNoNewline must be a format name: In scope SELECT formatRowNoNewline(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:12.698556 [ 7099 ] {ba61558f-9373-4963-9b88-7757bb41ae7f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fragment doesn't match: passed 3, should be 1: In scope SELECT fragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fragment(NULL, NULL, NULL);)
2024.11.04 23:20:12.700778 [ 7099 ] {ba61558f-9373-4963-9b88-7757bb41ae7f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fragment doesn't match: passed 3, should be 1: In scope SELECT fragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.730139 [ 7099 ] {03e26840-58cd-4763-a853-c99fa526cb7d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fromDaysSinceYearZero(NULL, NULL, NULL);)
2024.11.04 23:20:12.731919 [ 7099 ] {03e26840-58cd-4763-a853-c99fa526cb7d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.757224 [ 7099 ] {0928cb11-b221-45d8-8d41-c7fa5d21da6c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero32 doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fromDaysSinceYearZero32(NULL, NULL, NULL);)
2024.11.04 23:20:12.758385 [ 7099 ] {0928cb11-b221-45d8-8d41-c7fa5d21da6c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero32 doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.786066 [ 7099 ] {185946a1-5fbd-42eb-90e1-05b918170451} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDay(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fromModifiedJulianDay(NULL, NULL, NULL);)
2024.11.04 23:20:12.788006 [ 7099 ] {185946a1-5fbd-42eb-90e1-05b918170451} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDay(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.818229 [ 7099 ] {0da9f246-72de-4cc1-8ee8-dc1fe37739d9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDayOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fromModifiedJulianDayOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:12.820094 [ 7099 ] {0da9f246-72de-4cc1-8ee8-dc1fe37739d9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDayOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:12.899081 [ 7099 ] {b3f810bd-b44d-4be2-a610-f4acb6d2b014} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT fromUTCTimestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fromUTCTimestamp(NULL, NULL, NULL);)
2024.11.04 23:20:12.901256 [ 7099 ] {b3f810bd-b44d-4be2-a610-f4acb6d2b014} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT fromUTCTimestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.319262 [ 7099 ] {6ba8f6fa-31fc-4384-bf6f-cf9faf729b67} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT from_utc_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT from_utc_timestamp(NULL, NULL, NULL);)
2024.11.04 23:20:13.397928 [ 7099 ] {6ba8f6fa-31fc-4384-bf6f-cf9faf729b67} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT from_utc_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.432955 [ 7099 ] {80c05d9a-0a22-4b4e-8883-bfc48f04aae2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT fullHostName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fullHostName(NULL, NULL, NULL);)
2024.11.04 23:20:13.435104 [ 7099 ] {80c05d9a-0a22-4b4e-8883-bfc48f04aae2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT fullHostName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.469009 [ 7099 ] {cdc0d48b-63d6-4a31-ae39-1ae30cd94d05} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function fuzzBits doesn't match: passed 3, should be 2: In scope SELECT fuzzBits(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT fuzzBits(NULL, NULL, NULL);)
2024.11.04 23:20:13.471265 [ 7099 ] {cdc0d48b-63d6-4a31-ae39-1ae30cd94d05} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function fuzzBits doesn't match: passed 3, should be 2: In scope SELECT fuzzBits(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.652070 [ 7099 ] {22d72e88-b6fc-4a85-b25b-f43d4b6b3439} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function gcd doesn't match: passed 3, should be 2: In scope SELECT gcd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT gcd(NULL, NULL, NULL);)
2024.11.04 23:20:13.654263 [ 7099 ] {22d72e88-b6fc-4a85-b25b-f43d4b6b3439} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function gcd doesn't match: passed 3, should be 2: In scope SELECT gcd(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.692890 [ 3949 ] {a1e79526-91ff-4a51-aeb6-32f29a92f398} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47478) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:13.696816 [ 3949 ] {a1e79526-91ff-4a51-aeb6-32f29a92f398} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:13.697647 [ 7099 ] {f6d474e9-fc28-45c3-bad3-732398fba5f1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function generateRandomStructure doesn't match: passed 3, expected from 0 to 2: In scope SELECT generateRandomStructure(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT generateRandomStructure(NULL, NULL, NULL);)
2024.11.04 23:20:13.699870 [ 7099 ] {f6d474e9-fc28-45c3-bad3-732398fba5f1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function generateRandomStructure doesn't match: passed 3, expected from 0 to 2: In scope SELECT generateRandomStructure(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.739084 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:13.777695 [ 7099 ] {5c80f391-57a8-479b-833d-0d90cb657a2e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function generateUUIDv4 doesn't match: passed 3, should be 0 or 1.: In scope SELECT generateUUIDv4(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT generateUUIDv4(NULL, NULL, NULL);)
2024.11.04 23:20:13.779583 [ 7099 ] {5c80f391-57a8-479b-833d-0d90cb657a2e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function generateUUIDv4 doesn't match: passed 3, should be 0 or 1.: In scope SELECT generateUUIDv4(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.806638 [ 7099 ] {76141b2d-dd16-4323-9fc5-38b34b5d0a13} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function geoDistance doesn't match: passed 3, should be 4: In scope SELECT geoDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT geoDistance(NULL, NULL, NULL);)
2024.11.04 23:20:13.808179 [ 7099 ] {76141b2d-dd16-4323-9fc5-38b34b5d0a13} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function geoDistance doesn't match: passed 3, should be 4: In scope SELECT geoDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.913197 [ 7099 ] {4d008780-cda1-4d2f-abfe-94ec2c7e0532} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function geoToS2 doesn't match: passed 3, should be 2: In scope SELECT geoToS2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT geoToS2(NULL, NULL, NULL);)
2024.11.04 23:20:13.915041 [ 7099 ] {4d008780-cda1-4d2f-abfe-94ec2c7e0532} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function geoToS2 doesn't match: passed 3, should be 2: In scope SELECT geoToS2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:13.944985 [ 7099 ] {fd5b9b40-0f11-4b22-b4b9-ca54ead6ffd9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function geohashDecode doesn't match: passed 3, should be 1: In scope SELECT geohashDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT geohashDecode(NULL, NULL, NULL);)
2024.11.04 23:20:13.946875 [ 7099 ] {fd5b9b40-0f11-4b22-b4b9-ca54ead6ffd9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function geohashDecode doesn't match: passed 3, should be 1: In scope SELECT geohashDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.028104 [ 7099 ] {be367432-a6b5-4f16-9b1b-ab39f88bc26f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function geohashesInBox doesn't match: passed 3, should be 5: In scope SELECT geohashesInBox(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT geohashesInBox(NULL, NULL, NULL);)
2024.11.04 23:20:14.030461 [ 7099 ] {be367432-a6b5-4f16-9b1b-ab39f88bc26f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function geohashesInBox doesn't match: passed 3, should be 5: In scope SELECT geohashesInBox(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.033008 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:14.065439 [ 7099 ] {06547662-fb3b-4b2e-8c6f-cf4bf26f349a} <Error> executeQuery: Code: 446. DB::Exception: The function getClientHTTPHeader requires setting `allow_get_client_http_header` to be enabled. (FUNCTION_NOT_ALLOWED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getClientHTTPHeader(NULL, NULL, NULL);)
2024.11.04 23:20:14.067388 [ 7099 ] {06547662-fb3b-4b2e-8c6f-cf4bf26f349a} <Error> TCPHandler: Code: 446. DB::Exception: The function getClientHTTPHeader requires setting `allow_get_client_http_header` to be enabled. (FUNCTION_NOT_ALLOWED)
2024.11.04 23:20:14.096979 [ 7099 ] {22220848-fb2b-4f70-99e8-5b4120b2f2b8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function getMacro doesn't match: passed 3, should be 1: In scope SELECT getMacro(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getMacro(NULL, NULL, NULL);)
2024.11.04 23:20:14.098909 [ 7099 ] {22220848-fb2b-4f70-99e8-5b4120b2f2b8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function getMacro doesn't match: passed 3, should be 1: In scope SELECT getMacro(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.127907 [ 7099 ] {f89fbd3a-276a-43ca-8ca1-c3da5efee0b8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function getOSKernelVersion doesn't match: passed 3, should be 0: In scope SELECT getOSKernelVersion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getOSKernelVersion(NULL, NULL, NULL);)
2024.11.04 23:20:14.129943 [ 7099 ] {f89fbd3a-276a-43ca-8ca1-c3da5efee0b8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function getOSKernelVersion doesn't match: passed 3, should be 0: In scope SELECT getOSKernelVersion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.160207 [ 7099 ] {a864e1d0-e530-463d-a3d3-fb2cd38b02f7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function getServerPort doesn't match: passed 3, should be 1: In scope SELECT getServerPort(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getServerPort(NULL, NULL, NULL);)
2024.11.04 23:20:14.162375 [ 7099 ] {a864e1d0-e530-463d-a3d3-fb2cd38b02f7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function getServerPort doesn't match: passed 3, should be 1: In scope SELECT getServerPort(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.192138 [ 7099 ] {10202f29-cba2-4efa-b1b3-941663b67117} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function getSetting doesn't match: passed 3, should be 1: In scope SELECT getSetting(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getSetting(NULL, NULL, NULL);)
2024.11.04 23:20:14.194181 [ 7099 ] {10202f29-cba2-4efa-b1b3-941663b67117} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function getSetting doesn't match: passed 3, should be 1: In scope SELECT getSetting(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.263774 [ 7099 ] {72574338-e1ed-4119-a80b-2cbfc36c69e5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function getSizeOfEnumType doesn't match: passed 3, should be 1: In scope SELECT getSizeOfEnumType(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getSizeOfEnumType(NULL, NULL, NULL);)
2024.11.04 23:20:14.266094 [ 7099 ] {72574338-e1ed-4119-a80b-2cbfc36c69e5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function getSizeOfEnumType doesn't match: passed 3, should be 1: In scope SELECT getSizeOfEnumType(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.297536 [ 7099 ] {5cc4106b-8ced-4908-a8e8-73c9fbc1e09d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function getSubcolumn doesn't match: passed 3, should be 2: In scope SELECT getSubcolumn(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getSubcolumn(NULL, NULL, NULL);)
2024.11.04 23:20:14.299665 [ 7099 ] {5cc4106b-8ced-4908-a8e8-73c9fbc1e09d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function getSubcolumn doesn't match: passed 3, should be 2: In scope SELECT getSubcolumn(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.330481 [ 7099 ] {96ed8e11-9f80-4daf-8f1d-18f9a0ffafa3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function getTypeSerializationStreams doesn't match: passed 3, should be 1: In scope SELECT getTypeSerializationStreams(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT getTypeSerializationStreams(NULL, NULL, NULL);)
2024.11.04 23:20:14.332775 [ 7099 ] {96ed8e11-9f80-4daf-8f1d-18f9a0ffafa3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function getTypeSerializationStreams doesn't match: passed 3, should be 1: In scope SELECT getTypeSerializationStreams(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.363682 [ 7099 ] {8dca085e-d6f1-464d-9c48-6eab90da1a0c} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalIn(NULL, NULL, NULL);)
2024.11.04 23:20:14.366008 [ 7099 ] {8dca085e-d6f1-464d-9c48-6eab90da1a0c} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.397283 [ 7099 ] {e90b956e-1451-4dfc-b41a-acc2e9375802} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalInIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:14.399742 [ 7099 ] {e90b956e-1451-4dfc-b41a-acc2e9375802} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.489220 [ 7099 ] {6fbb29ee-a93f-40b0-a838-c9ab6886c790} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalNotIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalNotIn(NULL, NULL, NULL);)
2024.11.04 23:20:14.491584 [ 7099 ] {6fbb29ee-a93f-40b0-a838-c9ab6886c790} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalNotIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.522486 [ 7099 ] {2e8fc608-f3cd-40f6-a753-989d6d16a0b7} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalNotInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalNotInIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:14.524694 [ 7099 ] {2e8fc608-f3cd-40f6-a753-989d6d16a0b7} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalNotInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.561517 [ 7099 ] {d8eb90ee-e853-49cc-b33c-c86b133613ef} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalNotNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalNotNullIn(NULL, NULL, NULL);)
2024.11.04 23:20:14.563875 [ 7099 ] {d8eb90ee-e853-49cc-b33c-c86b133613ef} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalNotNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.595145 [ 7099 ] {d342cd87-65b3-45e8-863b-3956b3c51308} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalNotNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalNotNullInIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:14.597260 [ 7099 ] {d342cd87-65b3-45e8-863b-3956b3c51308} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalNotNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.627553 [ 7099 ] {de3d2a23-a2d0-45f3-a687-49760e710bfe} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalNullIn(NULL, NULL, NULL);)
2024.11.04 23:20:14.629777 [ 7099 ] {de3d2a23-a2d0-45f3-a687-49760e710bfe} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.659068 [ 7099 ] {db078e27-1c05-40c5-97e2-aedab83064fe} <Error> executeQuery: Code: 42. DB::Exception: Function 'globalNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalNullInIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:14.661075 [ 7099 ] {db078e27-1c05-40c5-97e2-aedab83064fe} <Error> TCPHandler: Code: 42. DB::Exception: Function 'globalNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.692991 [ 7099 ] {18903a97-fd3a-4af1-8060-6bc5b96d55cb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function globalVariable doesn't match: passed 3, should be 1: In scope SELECT globalVariable(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT globalVariable(NULL, NULL, NULL);)
2024.11.04 23:20:14.695072 [ 7099 ] {18903a97-fd3a-4af1-8060-6bc5b96d55cb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function globalVariable doesn't match: passed 3, should be 1: In scope SELECT globalVariable(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.742785 [ 3949 ] {a47f8f66-dc77-4f28-a3be-e64bbec4eafd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47492) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:14.746358 [ 3949 ] {a47f8f66-dc77-4f28-a3be-e64bbec4eafd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:14.777437 [ 7099 ] {48ce421b-95b0-416e-96d4-28b6fae01f9f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function greatCircleAngle doesn't match: passed 3, should be 4: In scope SELECT greatCircleAngle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT greatCircleAngle(NULL, NULL, NULL);)
2024.11.04 23:20:14.779415 [ 7099 ] {48ce421b-95b0-416e-96d4-28b6fae01f9f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function greatCircleAngle doesn't match: passed 3, should be 4: In scope SELECT greatCircleAngle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.872991 [ 7099 ] {eddc8a0c-8173-4244-829a-5b15884badd5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function greatCircleDistance doesn't match: passed 3, should be 4: In scope SELECT greatCircleDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT greatCircleDistance(NULL, NULL, NULL);)
2024.11.04 23:20:14.875024 [ 7099 ] {eddc8a0c-8173-4244-829a-5b15884badd5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function greatCircleDistance doesn't match: passed 3, should be 4: In scope SELECT greatCircleDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.907792 [ 7099 ] {7e575cdc-bc5b-4630-bb84-b3733a5aed94} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function greater doesn't match: passed 3, should be 2: In scope SELECT greater(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT greater(NULL, NULL, NULL);)
2024.11.04 23:20:14.909791 [ 7099 ] {7e575cdc-bc5b-4630-bb84-b3733a5aed94} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function greater doesn't match: passed 3, should be 2: In scope SELECT greater(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:14.942418 [ 7099 ] {6566f83e-2b1b-4c76-9338-02d3d94f1c11} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function greaterOrEquals doesn't match: passed 3, should be 2: In scope SELECT greaterOrEquals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT greaterOrEquals(NULL, NULL, NULL);)
2024.11.04 23:20:14.944890 [ 7099 ] {6566f83e-2b1b-4c76-9338-02d3d94f1c11} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function greaterOrEquals doesn't match: passed 3, should be 2: In scope SELECT greaterOrEquals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:15.196664 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:15.660119 [ 7099 ] {5e45a63f-def5-49b1-8a31-32a546f9c5f2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT groupArrayMovingAvg(NULL, NULL, NULL);)
2024.11.04 23:20:15.666717 [ 7099 ] {5e45a63f-def5-49b1-8a31-32a546f9c5f2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:15.710067 [ 3736 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:15.710238 [ 3736 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:15.793388 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:15.967604 [ 7099 ] {c9a222f9-e9ec-4dc9-ab6c-9f6d7c84f1a5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT groupArraySample(NULL, NULL, NULL);)
2024.11.04 23:20:15.974642 [ 7099 ] {c9a222f9-e9ec-4dc9-ab6c-9f6d7c84f1a5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:16.633478 [ 7099 ] {fcfe9e89-406d-4297-a9b1-6c450bc02d71} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT groupBitOr(NULL, NULL, NULL);)
2024.11.04 23:20:16.638626 [ 7099 ] {fcfe9e89-406d-4297-a9b1-6c450bc02d71} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:16.702679 [ 7099 ] {694186cf-0a29-48a8-8e48-259c05332f46} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT groupBitXor(NULL, NULL, NULL);)
2024.11.04 23:20:16.707672 [ 7099 ] {694186cf-0a29-48a8-8e48-259c05332f46} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:16.863508 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/arf/lvpjkuvjirovphoduqflvauukxknq, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65227093-678d-4145-b1d8-cac086ab72d9, upload has finished false.
2024.11.04 23:20:16.863675 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/arf/lvpjkuvjirovphoduqflvauukxknq, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65227093-678d-4145-b1d8-cac086ab72d9, upload has finished false
2024.11.04 23:20:16.871577 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/arf/lvpjkuvjirovphoduqflvauukxknq, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65227093-678d-4145-b1d8-cac086ab72d9, upload has finished false
2024.11.04 23:20:16.875760 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 10 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:16.883437 [ 7099 ] {5d69c115-f823-46c0-8f08-2aff6b9b2497} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT groupBitmapAnd(NULL, NULL, NULL);)
2024.11.04 23:20:16.979552 [ 7099 ] {5d69c115-f823-46c0-8f08-2aff6b9b2497} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:17.003334 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:17.035675 [ 3949 ] {384c0899-db5d-4ab6-b42a-61904d0e43a5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47506) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:17.039122 [ 3949 ] {384c0899-db5d-4ab6-b42a-61904d0e43a5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:17.291886 [ 7099 ] {010c90b8-b5ee-491d-8e40-bf230198db27} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT groupUniqArray(NULL, NULL, NULL);)
2024.11.04 23:20:17.296082 [ 7099 ] {010c90b8-b5ee-491d-8e40-bf230198db27} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:17.345789 [ 7099 ] {7dd92a40-366f-4def-a8bb-f37a7ec7670e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3CellAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaM2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3CellAreaM2(NULL, NULL, NULL);)
2024.11.04 23:20:17.347797 [ 7099 ] {7dd92a40-366f-4def-a8bb-f37a7ec7670e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3CellAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaM2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.377692 [ 7099 ] {00a139f6-1a51-4cbd-a34c-184b4cc133ae} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3CellAreaRads2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaRads2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3CellAreaRads2(NULL, NULL, NULL);)
2024.11.04 23:20:17.380059 [ 7099 ] {00a139f6-1a51-4cbd-a34c-184b4cc133ae} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3CellAreaRads2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaRads2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.413586 [ 7099 ] {495cc6e0-ddee-41e6-9de5-b4de0b031add} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3Distance doesn't match: passed 3, should be 2: In scope SELECT h3Distance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3Distance(NULL, NULL, NULL);)
2024.11.04 23:20:17.415931 [ 7099 ] {495cc6e0-ddee-41e6-9de5-b4de0b031add} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3Distance doesn't match: passed 3, should be 2: In scope SELECT h3Distance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.447158 [ 7099 ] {99df92db-3d7a-4e16-a45d-755a4351af47} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3EdgeAngle doesn't match: passed 3, should be 1: In scope SELECT h3EdgeAngle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3EdgeAngle(NULL, NULL, NULL);)
2024.11.04 23:20:17.449401 [ 7099 ] {99df92db-3d7a-4e16-a45d-755a4351af47} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3EdgeAngle doesn't match: passed 3, should be 1: In scope SELECT h3EdgeAngle(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.478429 [ 7099 ] {7eba7f53-81cd-400b-82d7-cd60916a4eb0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthKm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3EdgeLengthKm(NULL, NULL, NULL);)
2024.11.04 23:20:17.480333 [ 7099 ] {7eba7f53-81cd-400b-82d7-cd60916a4eb0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthKm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.515070 [ 7099 ] {7057cace-0a70-42ea-833d-a0db64b374c7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthM(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3EdgeLengthM(NULL, NULL, NULL);)
2024.11.04 23:20:17.517108 [ 7099 ] {7057cace-0a70-42ea-833d-a0db64b374c7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthM(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.547149 [ 7099 ] {0a6dbb04-8fdd-4003-b2c5-612c84eacd83} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthKm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ExactEdgeLengthKm(NULL, NULL, NULL);)
2024.11.04 23:20:17.549297 [ 7099 ] {0a6dbb04-8fdd-4003-b2c5-612c84eacd83} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthKm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.578767 [ 7099 ] {6fc828e3-a4b4-4506-bcd4-29ea6e53333f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthM(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ExactEdgeLengthM(NULL, NULL, NULL);)
2024.11.04 23:20:17.642326 [ 7099 ] {6fc828e3-a4b4-4506-bcd4-29ea6e53333f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthM(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.674029 [ 7099 ] {83c20212-e3ff-44ac-9d5f-9df8e28ce10b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthRads doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthRads(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ExactEdgeLengthRads(NULL, NULL, NULL);)
2024.11.04 23:20:17.676812 [ 7099 ] {83c20212-e3ff-44ac-9d5f-9df8e28ce10b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthRads doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthRads(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.709579 [ 7099 ] {41b0ff8c-e1c2-4f60-84dd-6efd4e7449a2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetBaseCell doesn't match: passed 3, should be 1: In scope SELECT h3GetBaseCell(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetBaseCell(NULL, NULL, NULL);)
2024.11.04 23:20:17.711815 [ 7099 ] {41b0ff8c-e1c2-4f60-84dd-6efd4e7449a2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetBaseCell doesn't match: passed 3, should be 1: In scope SELECT h3GetBaseCell(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.743780 [ 7099 ] {3d5c8432-6255-4437-b1ae-f7857d2aa3be} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetDestinationIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetDestinationIndexFromUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetDestinationIndexFromUnidirectionalEdge(NULL, NULL, NULL);)
2024.11.04 23:20:17.774244 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:17.775978 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=76, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:17.790540 [ 7099 ] {3d5c8432-6255-4437-b1ae-f7857d2aa3be} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetDestinationIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetDestinationIndexFromUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.872409 [ 7099 ] {4762ce05-6392-4d8c-9c5d-2dba9d2c8ab2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetFaces doesn't match: passed 3, should be 1: In scope SELECT h3GetFaces(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetFaces(NULL, NULL, NULL);)
2024.11.04 23:20:17.874543 [ 7099 ] {4762ce05-6392-4d8c-9c5d-2dba9d2c8ab2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetFaces doesn't match: passed 3, should be 1: In scope SELECT h3GetFaces(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.917334 [ 7099 ] {11011558-8218-42d0-a4c4-095b3d3dbd71} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetIndexesFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetIndexesFromUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetIndexesFromUnidirectionalEdge(NULL, NULL, NULL);)
2024.11.04 23:20:17.919789 [ 7099 ] {11011558-8218-42d0-a4c4-095b3d3dbd71} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetIndexesFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetIndexesFromUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.956887 [ 7099 ] {8a362e28-6da0-4ec2-b622-478eef01a52e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetOriginIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetOriginIndexFromUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetOriginIndexFromUnidirectionalEdge(NULL, NULL, NULL);)
2024.11.04 23:20:17.959134 [ 7099 ] {8a362e28-6da0-4ec2-b622-478eef01a52e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetOriginIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetOriginIndexFromUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:17.993516 [ 7099 ] {db841a81-e892-4cc1-8866-916f9636f5aa} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetPentagonIndexes doesn't match: passed 3, should be 1: In scope SELECT h3GetPentagonIndexes(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetPentagonIndexes(NULL, NULL, NULL);)
2024.11.04 23:20:17.995612 [ 7099 ] {db841a81-e892-4cc1-8866-916f9636f5aa} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetPentagonIndexes doesn't match: passed 3, should be 1: In scope SELECT h3GetPentagonIndexes(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.029829 [ 7099 ] {facea657-eaba-41a3-96de-52226b429c53} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetRes0Indexes doesn't match: passed 3, should be 0: In scope SELECT h3GetRes0Indexes(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetRes0Indexes(NULL, NULL, NULL);)
2024.11.04 23:20:18.032493 [ 7099 ] {facea657-eaba-41a3-96de-52226b429c53} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetRes0Indexes doesn't match: passed 3, should be 0: In scope SELECT h3GetRes0Indexes(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.063513 [ 7099 ] {4dc0c34d-c355-4377-bbfa-7e64b278fdba} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetResolution doesn't match: passed 3, should be 1: In scope SELECT h3GetResolution(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetResolution(NULL, NULL, NULL);)
2024.11.04 23:20:18.066221 [ 7099 ] {4dc0c34d-c355-4377-bbfa-7e64b278fdba} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetResolution doesn't match: passed 3, should be 1: In scope SELECT h3GetResolution(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.072578 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:18.099776 [ 7099 ] {b150e220-6987-4c39-b2cd-bb99ccaaad94} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdge doesn't match: passed 3, should be 2: In scope SELECT h3GetUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetUnidirectionalEdge(NULL, NULL, NULL);)
2024.11.04 23:20:18.102784 [ 7099 ] {b150e220-6987-4c39-b2cd-bb99ccaaad94} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdge doesn't match: passed 3, should be 2: In scope SELECT h3GetUnidirectionalEdge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.135324 [ 7099 ] {382b5071-4018-4f7a-9079-02376a5b9dcd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgeBoundary doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgeBoundary(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetUnidirectionalEdgeBoundary(NULL, NULL, NULL);)
2024.11.04 23:20:18.137418 [ 7099 ] {382b5071-4018-4f7a-9079-02376a5b9dcd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgeBoundary doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgeBoundary(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.220460 [ 7099 ] {5963d34e-71e8-4243-8537-7fbd7d44821d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgesFromHexagon doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgesFromHexagon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3GetUnidirectionalEdgesFromHexagon(NULL, NULL, NULL);)
2024.11.04 23:20:18.222411 [ 7099 ] {5963d34e-71e8-4243-8537-7fbd7d44821d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgesFromHexagon doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgesFromHexagon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.253972 [ 7099 ] {795dd02f-dcfc-4d4e-8fc3-fddb5ba3447e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3HexAreaKm2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaKm2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3HexAreaKm2(NULL, NULL, NULL);)
2024.11.04 23:20:18.256061 [ 7099 ] {795dd02f-dcfc-4d4e-8fc3-fddb5ba3447e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3HexAreaKm2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaKm2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.290423 [ 7099 ] {a2950643-ed50-45f1-a3ad-2edac5c7bdac} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3HexAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaM2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3HexAreaM2(NULL, NULL, NULL);)
2024.11.04 23:20:18.292955 [ 7099 ] {a2950643-ed50-45f1-a3ad-2edac5c7bdac} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3HexAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaM2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.359397 [ 7099 ] {bd645c3b-a3e0-4851-94f0-d0b96d531871} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3HexRing doesn't match: passed 3, should be 2: In scope SELECT h3HexRing(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3HexRing(NULL, NULL, NULL);)
2024.11.04 23:20:18.361796 [ 7099 ] {bd645c3b-a3e0-4851-94f0-d0b96d531871} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3HexRing doesn't match: passed 3, should be 2: In scope SELECT h3HexRing(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.398841 [ 7099 ] {bb68640c-d413-49e2-96f3-ca66bc80acb1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IndexesAreNeighbors doesn't match: passed 3, should be 2: In scope SELECT h3IndexesAreNeighbors(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3IndexesAreNeighbors(NULL, NULL, NULL);)
2024.11.04 23:20:18.401523 [ 7099 ] {bb68640c-d413-49e2-96f3-ca66bc80acb1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IndexesAreNeighbors doesn't match: passed 3, should be 2: In scope SELECT h3IndexesAreNeighbors(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.439389 [ 7099 ] {85a5655b-274b-4ddd-93ad-bd8295ebe1bb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IsPentagon doesn't match: passed 3, should be 1: In scope SELECT h3IsPentagon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3IsPentagon(NULL, NULL, NULL);)
2024.11.04 23:20:18.441841 [ 7099 ] {85a5655b-274b-4ddd-93ad-bd8295ebe1bb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IsPentagon doesn't match: passed 3, should be 1: In scope SELECT h3IsPentagon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.473143 [ 7099 ] {a290e45c-9622-4817-af0b-dea122e237a0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IsResClassIII doesn't match: passed 3, should be 1: In scope SELECT h3IsResClassIII(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3IsResClassIII(NULL, NULL, NULL);)
2024.11.04 23:20:18.475401 [ 7099 ] {a290e45c-9622-4817-af0b-dea122e237a0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IsResClassIII doesn't match: passed 3, should be 1: In scope SELECT h3IsResClassIII(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.508426 [ 7099 ] {9e497391-bc7c-431b-a68d-a1051d93d57e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IsValid doesn't match: passed 3, should be 1: In scope SELECT h3IsValid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3IsValid(NULL, NULL, NULL);)
2024.11.04 23:20:18.510618 [ 7099 ] {9e497391-bc7c-431b-a68d-a1051d93d57e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IsValid doesn't match: passed 3, should be 1: In scope SELECT h3IsValid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.542713 [ 7099 ] {a3c96c80-03ca-4ef4-ac6d-dd498b61e60e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3Line doesn't match: passed 3, should be 2: In scope SELECT h3Line(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3Line(NULL, NULL, NULL);)
2024.11.04 23:20:18.544928 [ 7099 ] {a3c96c80-03ca-4ef4-ac6d-dd498b61e60e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3Line doesn't match: passed 3, should be 2: In scope SELECT h3Line(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.574214 [ 7099 ] {ef6961ae-e214-4071-8ea6-93e205aefbad} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3NumHexagons doesn't match: passed 3, should be 1: In scope SELECT h3NumHexagons(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3NumHexagons(NULL, NULL, NULL);)
2024.11.04 23:20:18.576200 [ 7099 ] {ef6961ae-e214-4071-8ea6-93e205aefbad} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3NumHexagons doesn't match: passed 3, should be 1: In scope SELECT h3NumHexagons(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.610750 [ 7099 ] {15313ea0-d2cb-45e7-a1f3-32fa0a43bae8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3PointDistKm doesn't match: passed 3, should be 4: In scope SELECT h3PointDistKm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3PointDistKm(NULL, NULL, NULL);)
2024.11.04 23:20:18.613037 [ 7099 ] {15313ea0-d2cb-45e7-a1f3-32fa0a43bae8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3PointDistKm doesn't match: passed 3, should be 4: In scope SELECT h3PointDistKm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.644089 [ 7099 ] {29bb95d8-e38a-4cb1-895c-bf7b791643be} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3PointDistM doesn't match: passed 3, should be 4: In scope SELECT h3PointDistM(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3PointDistM(NULL, NULL, NULL);)
2024.11.04 23:20:18.646144 [ 7099 ] {29bb95d8-e38a-4cb1-895c-bf7b791643be} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3PointDistM doesn't match: passed 3, should be 4: In scope SELECT h3PointDistM(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.677733 [ 7099 ] {d502490e-3726-45cc-8871-7b17ac28b8c3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3PointDistRads doesn't match: passed 3, should be 4: In scope SELECT h3PointDistRads(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3PointDistRads(NULL, NULL, NULL);)
2024.11.04 23:20:18.679890 [ 7099 ] {d502490e-3726-45cc-8871-7b17ac28b8c3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3PointDistRads doesn't match: passed 3, should be 4: In scope SELECT h3PointDistRads(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.714211 [ 7099 ] {15e10a0f-5db3-4b83-910e-fc6ecb509d0e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToCenterChild doesn't match: passed 3, should be 2: In scope SELECT h3ToCenterChild(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ToCenterChild(NULL, NULL, NULL);)
2024.11.04 23:20:18.716420 [ 7099 ] {15e10a0f-5db3-4b83-910e-fc6ecb509d0e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToCenterChild doesn't match: passed 3, should be 2: In scope SELECT h3ToCenterChild(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.747646 [ 7099 ] {5c1c160d-3ad9-462c-9bef-164d11a84d07} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToChildren doesn't match: passed 3, should be 2: In scope SELECT h3ToChildren(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ToChildren(NULL, NULL, NULL);)
2024.11.04 23:20:18.841528 [ 7099 ] {5c1c160d-3ad9-462c-9bef-164d11a84d07} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToChildren doesn't match: passed 3, should be 2: In scope SELECT h3ToChildren(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.888279 [ 7099 ] {9c990a85-ae27-4b10-837d-8a160fb8003b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToGeo doesn't match: passed 3, should be 1: In scope SELECT h3ToGeo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ToGeo(NULL, NULL, NULL);)
2024.11.04 23:20:18.890699 [ 7099 ] {9c990a85-ae27-4b10-837d-8a160fb8003b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToGeo doesn't match: passed 3, should be 1: In scope SELECT h3ToGeo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.929534 [ 7099 ] {2f8ad356-14a9-48cd-8263-9868bea08c74} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToGeoBoundary doesn't match: passed 3, should be 1: In scope SELECT h3ToGeoBoundary(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ToGeoBoundary(NULL, NULL, NULL);)
2024.11.04 23:20:18.931820 [ 7099 ] {2f8ad356-14a9-48cd-8263-9868bea08c74} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToGeoBoundary doesn't match: passed 3, should be 1: In scope SELECT h3ToGeoBoundary(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:18.966450 [ 7099 ] {c436a49f-7c2a-4b4d-b7d2-d674c46964cd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToParent doesn't match: passed 3, should be 2: In scope SELECT h3ToParent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ToParent(NULL, NULL, NULL);)
2024.11.04 23:20:18.968870 [ 7099 ] {c436a49f-7c2a-4b4d-b7d2-d674c46964cd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToParent doesn't match: passed 3, should be 2: In scope SELECT h3ToParent(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.009691 [ 7099 ] {a8b7ae3e-b379-4c6d-a22e-90dc9255a82e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToString doesn't match: passed 3, should be 1: In scope SELECT h3ToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3ToString(NULL, NULL, NULL);)
2024.11.04 23:20:19.012175 [ 7099 ] {a8b7ae3e-b379-4c6d-a22e-90dc9255a82e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToString doesn't match: passed 3, should be 1: In scope SELECT h3ToString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.047213 [ 7099 ] {effc4bea-87e2-411b-b5a5-c745cad5ef96} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3UnidirectionalEdgeIsValid doesn't match: passed 3, should be 1: In scope SELECT h3UnidirectionalEdgeIsValid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3UnidirectionalEdgeIsValid(NULL, NULL, NULL);)
2024.11.04 23:20:19.049986 [ 7099 ] {effc4bea-87e2-411b-b5a5-c745cad5ef96} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3UnidirectionalEdgeIsValid doesn't match: passed 3, should be 1: In scope SELECT h3UnidirectionalEdgeIsValid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.090356 [ 7099 ] {d56d3f00-d226-44f5-bce7-f76d1119e5dd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function h3kRing doesn't match: passed 3, should be 2: In scope SELECT h3kRing(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT h3kRing(NULL, NULL, NULL);)
2024.11.04 23:20:19.187349 [ 3692 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded hard memory limit (14.54 GiB)
2024.11.04 23:20:19.189024 [ 7099 ] {d56d3f00-d226-44f5-bce7-f76d1119e5dd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3kRing doesn't match: passed 3, should be 2: In scope SELECT h3kRing(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.277561 [ 7099 ] {a25e3f02-66b1-45c2-adc5-972b4e19fde0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function has doesn't match: passed 3, should be 2: In scope SELECT has(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT has(NULL, NULL, NULL);)
2024.11.04 23:20:19.279552 [ 7099 ] {a25e3f02-66b1-45c2-adc5-972b4e19fde0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function has doesn't match: passed 3, should be 2: In scope SELECT has(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.309429 [ 7099 ] {b68c28bb-0cea-4a86-95ec-3ace205c3fe0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasAll doesn't match: passed 3, should be 2: In scope SELECT hasAll(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasAll(NULL, NULL, NULL);)
2024.11.04 23:20:19.311608 [ 7099 ] {b68c28bb-0cea-4a86-95ec-3ace205c3fe0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasAll doesn't match: passed 3, should be 2: In scope SELECT hasAll(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.340710 [ 7099 ] {ff8b86a0-6a25-455e-bb07-0eec25bb42a2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasAny doesn't match: passed 3, should be 2: In scope SELECT hasAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasAny(NULL, NULL, NULL);)
2024.11.04 23:20:19.342625 [ 7099 ] {ff8b86a0-6a25-455e-bb07-0eec25bb42a2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasAny doesn't match: passed 3, should be 2: In scope SELECT hasAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.414734 [ 7099 ] {49019ba0-8d42-4bbb-b28d-57d39c6922f4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequence doesn't match: passed 3, should be 2: In scope SELECT hasSubsequence(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasSubsequence(NULL, NULL, NULL);)
2024.11.04 23:20:19.416855 [ 7099 ] {49019ba0-8d42-4bbb-b28d-57d39c6922f4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequence doesn't match: passed 3, should be 2: In scope SELECT hasSubsequence(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.482243 [ 7099 ] {3eb56c1e-deaf-40fb-9d28-d4803fa69c3f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasSubsequenceCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:19.484155 [ 7099 ] {3eb56c1e-deaf-40fb-9d28-d4803fa69c3f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.520001 [ 7099 ] {82062c0f-db70-4962-821f-53f1e87f174a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasSubsequenceCaseInsensitiveUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:19.522254 [ 7099 ] {82062c0f-db70-4962-821f-53f1e87f174a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.552497 [ 7099 ] {fd1d0486-e04f-4988-9985-2ecfafeace6b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasSubsequenceUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:19.554567 [ 7099 ] {fd1d0486-e04f-4988-9985-2ecfafeace6b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.584819 [ 7099 ] {8de984a1-fa7d-44d2-b460-a5bbb53d7023} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubstr doesn't match: passed 3, should be 2: In scope SELECT hasSubstr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasSubstr(NULL, NULL, NULL);)
2024.11.04 23:20:19.586889 [ 7099 ] {8de984a1-fa7d-44d2-b460-a5bbb53d7023} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubstr doesn't match: passed 3, should be 2: In scope SELECT hasSubstr(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.618434 [ 7099 ] {339282fc-3043-43c1-ab63-ddbd68286868} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasThreadFuzzer doesn't match: passed 3, should be 0: In scope SELECT hasThreadFuzzer(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasThreadFuzzer(NULL, NULL, NULL);)
2024.11.04 23:20:19.620761 [ 7099 ] {339282fc-3043-43c1-ab63-ddbd68286868} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasThreadFuzzer doesn't match: passed 3, should be 0: In scope SELECT hasThreadFuzzer(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.700703 [ 7099 ] {1c015164-1380-4047-8b69-02a1dc6b3fb7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasToken doesn't match: passed 3, should be 2: In scope SELECT hasToken(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasToken(NULL, NULL, NULL);)
2024.11.04 23:20:19.703079 [ 7099 ] {1c015164-1380-4047-8b69-02a1dc6b3fb7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasToken doesn't match: passed 3, should be 2: In scope SELECT hasToken(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.733554 [ 7099 ] {5d2d2aae-6c42-400e-8af4-db4de4175c02} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasTokenCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:19.735910 [ 7099 ] {5d2d2aae-6c42-400e-8af4-db4de4175c02} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.846165 [ 7099 ] {e7adff78-1e80-468c-a6df-e9b5bbe41a1c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitiveOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitiveOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasTokenCaseInsensitiveOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:19.848543 [ 7099 ] {e7adff78-1e80-468c-a6df-e9b5bbe41a1c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitiveOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitiveOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.902567 [ 7099 ] {cf7c5011-bb49-483f-8ff2-9d14c7934370} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hasTokenOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hasTokenOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:19.904864 [ 7099 ] {cf7c5011-bb49-483f-8ff2-9d14c7934370} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasTokenOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:19.940627 [ 7099 ] {9b0a0739-ddb3-4ca9-89c3-f185be2e1cfb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hex doesn't match: passed 3, should be 1: In scope SELECT hex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hex(NULL, NULL, NULL);)
2024.11.04 23:20:19.942821 [ 7099 ] {9b0a0739-ddb3-4ca9-89c3-f185be2e1cfb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hex doesn't match: passed 3, should be 1: In scope SELECT hex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.041724 [ 7099 ] {9b3c32d7-6133-4fee-8840-b61323576815} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT histogram(NULL, NULL, NULL);)
2024.11.04 23:20:20.046625 [ 7099 ] {9b3c32d7-6133-4fee-8840-b61323576815} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:20.336753 [ 7099 ] {a1e8c26f-7109-4f0c-a61d-837869599216} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hostName(NULL, NULL, NULL);)
2024.11.04 23:20:20.339559 [ 7099 ] {a1e8c26f-7109-4f0c-a61d-837869599216} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.354363 [ 3949 ] {a57f73e9-38ac-4873-8e5c-37053326fe80} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47524) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:20.357489 [ 3949 ] {a57f73e9-38ac-4873-8e5c-37053326fe80} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:20.373322 [ 7099 ] {5327177f-0982-495a-88d4-fc5172833673} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostname(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hostname(NULL, NULL, NULL);)
2024.11.04 23:20:20.375474 [ 7099 ] {5327177f-0982-495a-88d4-fc5172833673} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostname(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.407914 [ 7099 ] {813e3222-fe54-4784-931a-b4c92a9630b7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function hypot doesn't match: passed 3, should be 2: In scope SELECT hypot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT hypot(NULL, NULL, NULL);)
2024.11.04 23:20:20.409994 [ 7099 ] {813e3222-fe54-4784-931a-b4c92a9630b7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function hypot doesn't match: passed 3, should be 2: In scope SELECT hypot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.443208 [ 7099 ] {83602b5d-cc55-41c2-aece-b2fb530cfccb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function identity doesn't match: passed 3, should be 1: In scope SELECT identity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT identity(NULL, NULL, NULL);)
2024.11.04 23:20:20.445412 [ 7099 ] {83602b5d-cc55-41c2-aece-b2fb530cfccb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function identity doesn't match: passed 3, should be 1: In scope SELECT identity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.502239 [ 7099 ] {5fa11f26-e577-4269-9026-df88c5bca8bb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function idnaDecode doesn't match: passed 3, should be 1: In scope SELECT idnaDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT idnaDecode(NULL, NULL, NULL);)
2024.11.04 23:20:20.504547 [ 7099 ] {5fa11f26-e577-4269-9026-df88c5bca8bb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function idnaDecode doesn't match: passed 3, should be 1: In scope SELECT idnaDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.534987 [ 7099 ] {2eedb3cc-58dd-4308-953a-1d1fe1e55499} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function idnaEncode doesn't match: passed 3, should be 1: In scope SELECT idnaEncode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT idnaEncode(NULL, NULL, NULL);)
2024.11.04 23:20:20.537259 [ 7099 ] {2eedb3cc-58dd-4308-953a-1d1fe1e55499} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function idnaEncode doesn't match: passed 3, should be 1: In scope SELECT idnaEncode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.608114 [ 7099 ] {4a332504-ed8b-47aa-9533-5289f375b880} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ifNotFinite doesn't match: passed 3, should be 2: In scope SELECT ifNotFinite(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ifNotFinite(NULL, NULL, NULL);)
2024.11.04 23:20:20.610118 [ 7099 ] {4a332504-ed8b-47aa-9533-5289f375b880} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ifNotFinite doesn't match: passed 3, should be 2: In scope SELECT ifNotFinite(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.638820 [ 7099 ] {6cfba676-7c5a-4e75-add8-ea376c2ee12f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ifNull doesn't match: passed 3, should be 2: In scope SELECT ifNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ifNull(NULL, NULL, NULL);)
2024.11.04 23:20:20.641097 [ 7099 ] {6cfba676-7c5a-4e75-add8-ea376c2ee12f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ifNull doesn't match: passed 3, should be 2: In scope SELECT ifNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.713389 [ 7099 ] {91de70e1-0d52-4402-9a61-63a8e33ab665} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ilike doesn't match: passed 3, should be 2: In scope SELECT ilike(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ilike(NULL, NULL, NULL);)
2024.11.04 23:20:20.715451 [ 7099 ] {91de70e1-0d52-4402-9a61-63a8e33ab665} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ilike doesn't match: passed 3, should be 2: In scope SELECT ilike(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.742618 [ 7099 ] {c0c3fecc-3497-4d4e-9993-5c3e3848233b} <Error> executeQuery: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT in(NULL, NULL, NULL);)
2024.11.04 23:20:20.744576 [ 7099 ] {c0c3fecc-3497-4d4e-9993-5c3e3848233b} <Error> TCPHandler: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.772123 [ 7099 ] {42bbc8e2-875c-4da9-a840-a57841edda2b} <Error> executeQuery: Code: 42. DB::Exception: Function 'inIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT inIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:20.774334 [ 7099 ] {42bbc8e2-875c-4da9-a840-a57841edda2b} <Error> TCPHandler: Code: 42. DB::Exception: Function 'inIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.853001 [ 7099 ] {3d85a6d8-4959-4afd-a9ac-ad4c8c01d26a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function indexOf doesn't match: passed 3, should be 2: In scope SELECT indexOf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT indexOf(NULL, NULL, NULL);)
2024.11.04 23:20:20.855307 [ 7099 ] {3d85a6d8-4959-4afd-a9ac-ad4c8c01d26a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function indexOf doesn't match: passed 3, should be 2: In scope SELECT indexOf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.945357 [ 7099 ] {a05feb70-5f70-4ce5-bd26-7bb4903c2abe} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function initcap doesn't match: passed 3, should be 1: In scope SELECT initcap(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT initcap(NULL, NULL, NULL);)
2024.11.04 23:20:20.947298 [ 7099 ] {a05feb70-5f70-4ce5-bd26-7bb4903c2abe} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function initcap doesn't match: passed 3, should be 1: In scope SELECT initcap(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:20.974963 [ 7099 ] {881bbc1b-033f-4d91-8f53-83747d1caeff} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function initcapUTF8 doesn't match: passed 3, should be 1: In scope SELECT initcapUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT initcapUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:20.991804 [ 7099 ] {881bbc1b-033f-4d91-8f53-83747d1caeff} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function initcapUTF8 doesn't match: passed 3, should be 1: In scope SELECT initcapUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.025469 [ 7099 ] {06a231bb-2f1b-4f17-9783-12609526af15} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initialQueryID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT initialQueryID(NULL, NULL, NULL);)
2024.11.04 23:20:21.027605 [ 7099 ] {06a231bb-2f1b-4f17-9783-12609526af15} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initialQueryID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.060931 [ 7099 ] {87a69baa-65a4-4fab-9814-858e9d4a05fe} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initial_query_id(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT initial_query_id(NULL, NULL, NULL);)
2024.11.04 23:20:21.063144 [ 7099 ] {87a69baa-65a4-4fab-9814-858e9d4a05fe} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initial_query_id(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.092485 [ 7099 ] {aeb82883-fe93-4cbe-be7a-f778c5a7323f} <Error> executeQuery: Code: 43. DB::Exception: First argument for function initializeAggregation must be constant string: name of aggregate function.: In scope SELECT initializeAggregation(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT initializeAggregation(NULL, NULL, NULL);)
2024.11.04 23:20:21.094575 [ 7099 ] {aeb82883-fe93-4cbe-be7a-f778c5a7323f} <Error> TCPHandler: Code: 43. DB::Exception: First argument for function initializeAggregation must be constant string: name of aggregate function.: In scope SELECT initializeAggregation(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:21.213312 [ 7099 ] {b3e238c0-aad8-42c1-9776-1528cbe00b2c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function intDiv doesn't match: passed 3, should be 2: In scope SELECT intDiv(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT intDiv(NULL, NULL, NULL);)
2024.11.04 23:20:21.215659 [ 7099 ] {b3e238c0-aad8-42c1-9776-1528cbe00b2c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function intDiv doesn't match: passed 3, should be 2: In scope SELECT intDiv(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.245232 [ 7099 ] {b74623a4-afa5-4be7-9951-f0db77920553} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function intDivOrZero doesn't match: passed 3, should be 2: In scope SELECT intDivOrZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT intDivOrZero(NULL, NULL, NULL);)
2024.11.04 23:20:21.247058 [ 7099 ] {b74623a4-afa5-4be7-9951-f0db77920553} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function intDivOrZero doesn't match: passed 3, should be 2: In scope SELECT intDivOrZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.285072 [ 7099 ] {45a6a1df-2936-4518-89c4-c5c3df092cb3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function intExp10 doesn't match: passed 3, should be 1: In scope SELECT intExp10(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT intExp10(NULL, NULL, NULL);)
2024.11.04 23:20:21.294981 [ 7099 ] {45a6a1df-2936-4518-89c4-c5c3df092cb3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function intExp10 doesn't match: passed 3, should be 1: In scope SELECT intExp10(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.324400 [ 7099 ] {61ccb2ef-3b4d-47dc-bc4d-99d29ab6df2f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function intExp2 doesn't match: passed 3, should be 1: In scope SELECT intExp2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT intExp2(NULL, NULL, NULL);)
2024.11.04 23:20:21.326402 [ 7099 ] {61ccb2ef-3b4d-47dc-bc4d-99d29ab6df2f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function intExp2 doesn't match: passed 3, should be 1: In scope SELECT intExp2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.354697 [ 7099 ] {4bcd96e1-6531-4bc0-8ef8-802c34cd9f83} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function intHash32 doesn't match: passed 3, should be 1: In scope SELECT intHash32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT intHash32(NULL, NULL, NULL);)
2024.11.04 23:20:21.356763 [ 7099 ] {4bcd96e1-6531-4bc0-8ef8-802c34cd9f83} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function intHash32 doesn't match: passed 3, should be 1: In scope SELECT intHash32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.384530 [ 7099 ] {7f27a05f-be38-452f-9a15-8ab96e7b82a6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function intHash64 doesn't match: passed 3, should be 1: In scope SELECT intHash64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT intHash64(NULL, NULL, NULL);)
2024.11.04 23:20:21.386850 [ 7099 ] {7f27a05f-be38-452f-9a15-8ab96e7b82a6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function intHash64 doesn't match: passed 3, should be 1: In scope SELECT intHash64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.526339 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_114_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=106). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:21.527787 [ 7099 ] {c4045233-beb0-45d2-9bc7-b8a5d638c2b3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isConstant doesn't match: passed 3, should be 1: In scope SELECT isConstant(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isConstant(NULL, NULL, NULL);)
2024.11.04 23:20:21.529878 [ 7099 ] {c4045233-beb0-45d2-9bc7-b8a5d638c2b3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isConstant doesn't match: passed 3, should be 1: In scope SELECT isConstant(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.606695 [ 7099 ] {db5e28c3-1c34-488d-81ae-7580e4b6383b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isFinite doesn't match: passed 3, should be 1: In scope SELECT isFinite(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isFinite(NULL, NULL, NULL);)
2024.11.04 23:20:21.608762 [ 7099 ] {db5e28c3-1c34-488d-81ae-7580e4b6383b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isFinite doesn't match: passed 3, should be 1: In scope SELECT isFinite(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.636664 [ 7099 ] {e49a40f8-2be5-403f-887c-d7e7c5507f8a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isIPAddressInRange doesn't match: passed 3, should be 2: In scope SELECT isIPAddressInRange(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isIPAddressInRange(NULL, NULL, NULL);)
2024.11.04 23:20:21.638594 [ 7099 ] {e49a40f8-2be5-403f-887c-d7e7c5507f8a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isIPAddressInRange doesn't match: passed 3, should be 2: In scope SELECT isIPAddressInRange(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.666085 [ 7099 ] {4952751b-05fd-415f-843c-778a628760fa} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isIPv4String doesn't match: passed 3, should be 1: In scope SELECT isIPv4String(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isIPv4String(NULL, NULL, NULL);)
2024.11.04 23:20:21.668139 [ 7099 ] {4952751b-05fd-415f-843c-778a628760fa} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isIPv4String doesn't match: passed 3, should be 1: In scope SELECT isIPv4String(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.697637 [ 7099 ] {6d8012d8-084c-49f9-bcd6-f4670a586add} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isIPv6String doesn't match: passed 3, should be 1: In scope SELECT isIPv6String(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isIPv6String(NULL, NULL, NULL);)
2024.11.04 23:20:21.742094 [ 7099 ] {6d8012d8-084c-49f9-bcd6-f4670a586add} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isIPv6String doesn't match: passed 3, should be 1: In scope SELECT isIPv6String(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.770149 [ 7099 ] {5fae6316-096f-4592-be85-5e5ee6d6b5ce} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isInfinite doesn't match: passed 3, should be 1: In scope SELECT isInfinite(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isInfinite(NULL, NULL, NULL);)
2024.11.04 23:20:21.772165 [ 7099 ] {5fae6316-096f-4592-be85-5e5ee6d6b5ce} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isInfinite doesn't match: passed 3, should be 1: In scope SELECT isInfinite(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.802824 [ 7099 ] {d2db6dbd-e81e-4c08-8340-f30ed99bd75c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isNaN doesn't match: passed 3, should be 1: In scope SELECT isNaN(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isNaN(NULL, NULL, NULL);)
2024.11.04 23:20:21.805307 [ 7099 ] {d2db6dbd-e81e-4c08-8340-f30ed99bd75c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNaN doesn't match: passed 3, should be 1: In scope SELECT isNaN(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.838586 [ 7099 ] {0dd31b27-9b20-4c6c-bd0f-3f426a9e2f69} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isNotDistinctFrom doesn't match: passed 3, should be 2: In scope SELECT isNotDistinctFrom(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isNotDistinctFrom(NULL, NULL, NULL);)
2024.11.04 23:20:21.840988 [ 7099 ] {0dd31b27-9b20-4c6c-bd0f-3f426a9e2f69} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNotDistinctFrom doesn't match: passed 3, should be 2: In scope SELECT isNotDistinctFrom(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.947454 [ 7099 ] {50d3ee0a-aed9-448c-8029-ca3618c8396e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isNotNull doesn't match: passed 3, should be 1: In scope SELECT isNotNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isNotNull(NULL, NULL, NULL);)
2024.11.04 23:20:21.949864 [ 7099 ] {50d3ee0a-aed9-448c-8029-ca3618c8396e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNotNull doesn't match: passed 3, should be 1: In scope SELECT isNotNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:21.982493 [ 7099 ] {c3de4abf-e5ee-4092-8b1b-68db9a778cc4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isNull doesn't match: passed 3, should be 1: In scope SELECT isNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isNull(NULL, NULL, NULL);)
2024.11.04 23:20:21.984853 [ 7099 ] {c3de4abf-e5ee-4092-8b1b-68db9a778cc4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNull doesn't match: passed 3, should be 1: In scope SELECT isNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.015544 [ 7099 ] {c58a4700-32dc-4363-9408-6999711853bb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isNullable doesn't match: passed 3, should be 1: In scope SELECT isNullable(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isNullable(NULL, NULL, NULL);)
2024.11.04 23:20:22.017665 [ 7099 ] {c58a4700-32dc-4363-9408-6999711853bb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNullable doesn't match: passed 3, should be 1: In scope SELECT isNullable(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.046750 [ 7099 ] {dd74fe9f-eb83-4a5b-9951-e49589e907a1} <Error> executeQuery: Code: 42. DB::Exception: Function isValidJSON needs exactly one argument: In scope SELECT isValidJSON(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isValidJSON(NULL, NULL, NULL);)
2024.11.04 23:20:22.049196 [ 7099 ] {dd74fe9f-eb83-4a5b-9951-e49589e907a1} <Error> TCPHandler: Code: 42. DB::Exception: Function isValidJSON needs exactly one argument: In scope SELECT isValidJSON(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.086774 [ 7099 ] {803d6039-cfa5-4a03-8f1f-f371b829e9da} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT isValidUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isValidUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:22.089268 [ 7099 ] {803d6039-cfa5-4a03-8f1f-f371b829e9da} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT isValidUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.131317 [ 7099 ] {77b25342-0a89-4f96-be34-ef26936c2ca5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function isZeroOrNull doesn't match: passed 3, should be 1: In scope SELECT isZeroOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT isZeroOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:22.133505 [ 7099 ] {77b25342-0a89-4f96-be34-ef26936c2ca5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function isZeroOrNull doesn't match: passed 3, should be 1: In scope SELECT isZeroOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.167304 [ 7099 ] {6ae50c7a-50f8-49ce-ab9e-834e50d53c9e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function jaroSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroSimilarity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT jaroSimilarity(NULL, NULL, NULL);)
2024.11.04 23:20:22.169278 [ 7099 ] {6ae50c7a-50f8-49ce-ab9e-834e50d53c9e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function jaroSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroSimilarity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.191384 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_104_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:22.242726 [ 7099 ] {9f9b3f7d-0e70-4cda-8b69-72723034c7a3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function jaroWinklerSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroWinklerSimilarity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT jaroWinklerSimilarity(NULL, NULL, NULL);)
2024.11.04 23:20:22.244881 [ 7099 ] {9f9b3f7d-0e70-4cda-8b69-72723034c7a3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function jaroWinklerSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroWinklerSimilarity(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.450433 [ 7099 ] {eaaa2bb8-6226-4a1e-aeac-6c10065cda6f} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGet(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT joinGet(NULL, NULL, NULL);)
2024.11.04 23:20:22.452637 [ 7099 ] {eaaa2bb8-6226-4a1e-aeac-6c10065cda6f} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGet(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:22.483434 [ 7099 ] {ae5399f4-add1-4c89-9bdc-f6d47280d6dd} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGetOrNull(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT joinGetOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:22.485408 [ 7099 ] {ae5399f4-add1-4c89-9bdc-f6d47280d6dd} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGetOrNull(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:22.531001 [ 3949 ] {18569a2e-a817-4573-955f-5d944dd4a1f8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47550) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:22.534524 [ 3949 ] {18569a2e-a817-4573-955f-5d944dd4a1f8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:22.567763 [ 7099 ] {3ec1db38-1d82-4df7-bc57-4ebe64c6331a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function jumpConsistentHash doesn't match: passed 3, should be 2: In scope SELECT jumpConsistentHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT jumpConsistentHash(NULL, NULL, NULL);)
2024.11.04 23:20:22.655374 [ 7099 ] {3ec1db38-1d82-4df7-bc57-4ebe64c6331a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function jumpConsistentHash doesn't match: passed 3, should be 2: In scope SELECT jumpConsistentHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:22.786962 [ 7099 ] {f033dae0-4191-4a76-8fc9-9c3a9f810658} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT kostikConsistentHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT kostikConsistentHash(NULL, NULL, NULL);)
2024.11.04 23:20:22.789430 [ 7099 ] {f033dae0-4191-4a76-8fc9-9c3a9f810658} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT kostikConsistentHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.038052 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_105_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:23.130563 [ 7099 ] {88f98412-6f52-4fae-9f3c-b01854b35e8d} <Error> executeQuery: Code: 36. DB::Exception: Offset must be an integer, 'Nullable(Nothing)' given. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lagInFrame(NULL, NULL, NULL);)
2024.11.04 23:20:23.132501 [ 7099 ] {88f98412-6f52-4fae-9f3c-b01854b35e8d} <Error> TCPHandler: Code: 36. DB::Exception: Offset must be an integer, 'Nullable(Nothing)' given. (BAD_ARGUMENTS)
2024.11.04 23:20:23.283298 [ 7099 ] {e46e60bc-c2d0-49c1-a215-62d9b3788010} <Error> executeQuery: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT last_value_respect_nulls(NULL, NULL, NULL);)
2024.11.04 23:20:23.285705 [ 7099 ] {e46e60bc-c2d0-49c1-a215-62d9b3788010} <Error> TCPHandler: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.314168 [ 7099 ] {878ac317-13f2-4782-9dc7-52f2205a6bab} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lcase(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lcase(NULL, NULL, NULL);)
2024.11.04 23:20:23.316590 [ 7099 ] {878ac317-13f2-4782-9dc7-52f2205a6bab} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lcase(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.343925 [ 7099 ] {be27e954-82fe-46d4-922c-69bb51db68c1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lcm doesn't match: passed 3, should be 2: In scope SELECT lcm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lcm(NULL, NULL, NULL);)
2024.11.04 23:20:23.345930 [ 7099 ] {be27e954-82fe-46d4-922c-69bb51db68c1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lcm doesn't match: passed 3, should be 2: In scope SELECT lcm(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.374533 [ 7099 ] {85768686-d48e-45f6-a392-7784bafe33b6} <Error> executeQuery: Code: 36. DB::Exception: Offset must be an integer, 'Nullable(Nothing)' given. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT leadInFrame(NULL, NULL, NULL);)
2024.11.04 23:20:23.376669 [ 7099 ] {85768686-d48e-45f6-a392-7784bafe33b6} <Error> TCPHandler: Code: 36. DB::Exception: Offset must be an integer, 'Nullable(Nothing)' given. (BAD_ARGUMENTS)
2024.11.04 23:20:23.447702 [ 7099 ] {a097664e-bb87-4844-9979-1505f411a869} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function left doesn't match: passed 3, should be 2: In scope SELECT left(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT left(NULL, NULL, NULL);)
2024.11.04 23:20:23.449854 [ 7099 ] {a097664e-bb87-4844-9979-1505f411a869} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function left doesn't match: passed 3, should be 2: In scope SELECT left(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.456836 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_105_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:23.624445 [ 7099 ] {1a0ef563-31e4-4c77-9fa4-6979c4373b77} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function leftUTF8 doesn't match: passed 3, should be 2: In scope SELECT leftUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT leftUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:23.626463 [ 7099 ] {1a0ef563-31e4-4c77-9fa4-6979c4373b77} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function leftUTF8 doesn't match: passed 3, should be 2: In scope SELECT leftUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.652472 [ 7099 ] {88584593-6800-4a9c-9c30-7304944983e0} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'lemmatize' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lemmatize(NULL, NULL, NULL);)
2024.11.04 23:20:23.654332 [ 7099 ] {88584593-6800-4a9c-9c30-7304944983e0} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'lemmatize' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:23.683214 [ 7099 ] {e4fa9a8f-b2bb-4acc-a3dc-ec0f8f4310b3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT length(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT length(NULL, NULL, NULL);)
2024.11.04 23:20:23.685875 [ 7099 ] {e4fa9a8f-b2bb-4acc-a3dc-ec0f8f4310b3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT length(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.711846 [ 7099 ] {58f8a629-e090-42f4-92af-4efd2341cf90} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT lengthUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lengthUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:23.713815 [ 7099 ] {58f8a629-e090-42f4-92af-4efd2341cf90} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT lengthUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.744102 [ 7099 ] {744b1cf4-b3d6-49d0-83fa-e239b6b279bb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function less doesn't match: passed 3, should be 2: In scope SELECT less(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT less(NULL, NULL, NULL);)
2024.11.04 23:20:23.747888 [ 7099 ] {744b1cf4-b3d6-49d0-83fa-e239b6b279bb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function less doesn't match: passed 3, should be 2: In scope SELECT less(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.776035 [ 7099 ] {f0044027-1a39-4c48-a908-1f9a20f0f942} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lessOrEquals doesn't match: passed 3, should be 2: In scope SELECT lessOrEquals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lessOrEquals(NULL, NULL, NULL);)
2024.11.04 23:20:23.778318 [ 7099 ] {f0044027-1a39-4c48-a908-1f9a20f0f942} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lessOrEquals doesn't match: passed 3, should be 2: In scope SELECT lessOrEquals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.804953 [ 7099 ] {aee37a1f-f964-4eb6-a63e-c58fcd344c99} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT levenshteinDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT levenshteinDistance(NULL, NULL, NULL);)
2024.11.04 23:20:23.806832 [ 7099 ] {aee37a1f-f964-4eb6-a63e-c58fcd344c99} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT levenshteinDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.836676 [ 7099 ] {39d0c825-f58d-4177-b671-c6ebfc191ab2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lgamma doesn't match: passed 3, should be 1: In scope SELECT lgamma(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lgamma(NULL, NULL, NULL);)
2024.11.04 23:20:23.839373 [ 7099 ] {39d0c825-f58d-4177-b671-c6ebfc191ab2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lgamma doesn't match: passed 3, should be 1: In scope SELECT lgamma(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.878021 [ 7099 ] {d5aa0483-e792-4a75-bf43-cd34457bb3cb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function like doesn't match: passed 3, should be 2: In scope SELECT like(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT like(NULL, NULL, NULL);)
2024.11.04 23:20:23.880728 [ 7099 ] {d5aa0483-e792-4a75-bf43-cd34457bb3cb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function like doesn't match: passed 3, should be 2: In scope SELECT like(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:23.958119 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_110_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:24.002071 [ 7099 ] {de55dd9d-3353-418c-8af2-d5237e1b29d4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT ln(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ln(NULL, NULL, NULL);)
2024.11.04 23:20:24.004269 [ 7099 ] {de55dd9d-3353-418c-8af2-d5237e1b29d4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT ln(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.071009 [ 7099 ] {1d6e1c3c-8385-46e4-86bf-15f1c30811ec} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT log(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT log(NULL, NULL, NULL);)
2024.11.04 23:20:24.073016 [ 7099 ] {1d6e1c3c-8385-46e4-86bf-15f1c30811ec} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT log(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.194957 [ 7099 ] {0702ffad-86b5-4c25-90db-c85d7940dfc0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function log10 doesn't match: passed 3, should be 1: In scope SELECT log10(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT log10(NULL, NULL, NULL);)
2024.11.04 23:20:24.196967 [ 7099 ] {0702ffad-86b5-4c25-90db-c85d7940dfc0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function log10 doesn't match: passed 3, should be 1: In scope SELECT log10(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.223054 [ 7099 ] {3de1727a-f3bc-4b30-bf73-fecfef99377e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function log1p doesn't match: passed 3, should be 1: In scope SELECT log1p(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT log1p(NULL, NULL, NULL);)
2024.11.04 23:20:24.225169 [ 7099 ] {3de1727a-f3bc-4b30-bf73-fecfef99377e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function log1p doesn't match: passed 3, should be 1: In scope SELECT log1p(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.250657 [ 7099 ] {7707c86e-c889-44ca-9e7c-1d909f41ce25} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function log2 doesn't match: passed 3, should be 1: In scope SELECT log2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT log2(NULL, NULL, NULL);)
2024.11.04 23:20:24.252512 [ 7099 ] {7707c86e-c889-44ca-9e7c-1d909f41ce25} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function log2 doesn't match: passed 3, should be 1: In scope SELECT log2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.344675 [ 7099 ] {12481277-27da-45e8-8c81-326d7a615d1e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function logTrace doesn't match: passed 3, should be 1: In scope SELECT logTrace(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT logTrace(NULL, NULL, NULL);)
2024.11.04 23:20:24.346957 [ 7099 ] {12481277-27da-45e8-8c81-326d7a615d1e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function logTrace doesn't match: passed 3, should be 1: In scope SELECT logTrace(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.375944 [ 7099 ] {070b0b91-71dc-46b9-b589-ef7e6a65253f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lowCardinalityIndices doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityIndices(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lowCardinalityIndices(NULL, NULL, NULL);)
2024.11.04 23:20:24.378064 [ 7099 ] {070b0b91-71dc-46b9-b589-ef7e6a65253f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lowCardinalityIndices doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityIndices(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.407607 [ 7099 ] {936190e8-2f5d-4afb-910a-ef74be2e8c50} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lowCardinalityKeys doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityKeys(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lowCardinalityKeys(NULL, NULL, NULL);)
2024.11.04 23:20:24.409985 [ 7099 ] {936190e8-2f5d-4afb-910a-ef74be2e8c50} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lowCardinalityKeys doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityKeys(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.437132 [ 7099 ] {e4588202-8231-41f1-99f8-780e9401b5f6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lower(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lower(NULL, NULL, NULL);)
2024.11.04 23:20:24.439168 [ 7099 ] {e4588202-8231-41f1-99f8-780e9401b5f6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lower(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.465830 [ 7099 ] {8f7d5da4-28ee-4400-bb9a-8b752a33989e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function lowerUTF8 doesn't match: passed 3, should be 1: In scope SELECT lowerUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT lowerUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:24.467852 [ 7099 ] {8f7d5da4-28ee-4400-bb9a-8b752a33989e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function lowerUTF8 doesn't match: passed 3, should be 1: In scope SELECT lowerUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.890324 [ 7099 ] {7f6d2b7a-f87a-4cbb-8d79-7d0da637f9a7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mannWhitneyUTest(NULL, NULL, NULL);)
2024.11.04 23:20:24.895536 [ 7099 ] {7f6d2b7a-f87a-4cbb-8d79-7d0da637f9a7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:24.928024 [ 7099 ] {d685f746-68ee-475a-a2d5-983f6f5af4a0} <Error> executeQuery: Code: 42. DB::Exception: Function map requires even number of arguments, but 3 given: In scope SELECT map(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT map(NULL, NULL, NULL);)
2024.11.04 23:20:24.930400 [ 7099 ] {d685f746-68ee-475a-a2d5-983f6f5af4a0} <Error> TCPHandler: Code: 42. DB::Exception: Function map requires even number of arguments, but 3 given: In scope SELECT map(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:24.969086 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_114_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=35). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:25.237429 [ 7099 ] {6135c669-2601-4e7b-9d52-abf8b49e782e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function mapContains doesn't match: passed 3, should be 2: In scope SELECT mapContains(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mapContains(NULL, NULL, NULL);)
2024.11.04 23:20:25.239372 [ 7099 ] {6135c669-2601-4e7b-9d52-abf8b49e782e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapContains doesn't match: passed 3, should be 2: In scope SELECT mapContains(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:25.538315 [ 7099 ] {6afc8efe-d087-43c5-b5aa-29fcf7b59c4d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT mapFromArrays(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mapFromArrays(NULL, NULL, NULL);)
2024.11.04 23:20:25.540800 [ 7099 ] {6afc8efe-d087-43c5-b5aa-29fcf7b59c4d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT mapFromArrays(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:25.605794 [ 7099 ] {a881a6a6-9c0d-4cbf-b65c-4b5f9ab62711} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function mapKeys doesn't match: passed 3, should be 1: In scope SELECT mapKeys(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mapKeys(NULL, NULL, NULL);)
2024.11.04 23:20:25.607773 [ 7099 ] {a881a6a6-9c0d-4cbf-b65c-4b5f9ab62711} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapKeys doesn't match: passed 3, should be 1: In scope SELECT mapKeys(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:25.926211 [ 7099 ] {b92b7899-b2d7-4687-9521-f672349d1f19} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function mapUpdate doesn't match: passed 3, should be 2: In scope SELECT mapUpdate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mapUpdate(NULL, NULL, NULL);)
2024.11.04 23:20:25.928143 [ 7099 ] {b92b7899-b2d7-4687-9521-f672349d1f19} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapUpdate doesn't match: passed 3, should be 2: In scope SELECT mapUpdate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:25.959744 [ 7099 ] {d2896f9b-30b5-4492-9bc7-a844dd1d2e32} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function mapValues doesn't match: passed 3, should be 1: In scope SELECT mapValues(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mapValues(NULL, NULL, NULL);)
2024.11.04 23:20:25.961895 [ 7099 ] {d2896f9b-30b5-4492-9bc7-a844dd1d2e32} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapValues doesn't match: passed 3, should be 1: In scope SELECT mapValues(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:25.973112 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:25.993779 [ 7099 ] {ed4992e0-e81d-4696-ad3d-cf98c50df954} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT match(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT match(NULL, NULL, NULL);)
2024.11.04 23:20:25.996205 [ 7099 ] {ed4992e0-e81d-4696-ad3d-cf98c50df954} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT match(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:26.031993 [ 7099 ] {74c11ccb-5f27-4329-bcb5-d336c409e3b1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function materialize doesn't match: passed 3, should be 1: In scope SELECT materialize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT materialize(NULL, NULL, NULL);)
2024.11.04 23:20:26.039926 [ 7099 ] {74c11ccb-5f27-4329-bcb5-d336c409e3b1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function materialize doesn't match: passed 3, should be 1: In scope SELECT materialize(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:26.091496 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_110_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:26.122730 [ 7099 ] {dfa13f9d-2fb2-407d-a84e-212234b14bee} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT max(NULL, NULL, NULL);)
2024.11.04 23:20:26.127801 [ 7099 ] {dfa13f9d-2fb2-407d-a84e-212234b14bee} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:26.154109 [ 7099 ] {966281bf-2a6b-4b6d-9d99-a6056e7a619e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function max2 doesn't match: passed 3, should be 2: In scope SELECT max2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT max2(NULL, NULL, NULL);)
2024.11.04 23:20:26.156088 [ 7099 ] {966281bf-2a6b-4b6d-9d99-a6056e7a619e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function max2 doesn't match: passed 3, should be 2: In scope SELECT max2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:26.207640 [ 7099 ] {f49af74e-db11-41a5-b592-0574d4d203d9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT maxIntersections(NULL, NULL, NULL);)
2024.11.04 23:20:26.213589 [ 7099 ] {f49af74e-db11-41a5-b592-0574d4d203d9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:26.303883 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/nby/nwzfgpbpstskoxhryzylgmbylaody, total size 318767104, count 318767104, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65d5ef3a-13ba-48a2-ac01-0884b1cfd3d3, upload has finished false.
2024.11.04 23:20:26.304097 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/nby/nwzfgpbpstskoxhryzylgmbylaody, total size 318767104, count 318767104, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65d5ef3a-13ba-48a2-ac01-0884b1cfd3d3, upload has finished false
2024.11.04 23:20:26.360686 [ 3693 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/nby/nwzfgpbpstskoxhryzylgmbylaody, total size 318767104, count 318767104, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65d5ef3a-13ba-48a2-ac01-0884b1cfd3d3, upload has finished false
2024.11.04 23:20:26.365946 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 87 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:26.514190 [ 7099 ] {d6118ef3-5ef3-445d-b43f-d86c9c48d48f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT maxMappedArrays(NULL, NULL, NULL);)
2024.11.04 23:20:26.520142 [ 7099 ] {d6118ef3-5ef3-445d-b43f-d86c9c48d48f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=31, jobs=31). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:26.574620 [ 7099 ] {e47621b4-5b86-4cac-9999-2c4fa5af00d4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT meanZTest(NULL, NULL, NULL);)
2024.11.04 23:20:26.578580 [ 7099 ] {e47621b4-5b86-4cac-9999-2c4fa5af00d4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:26.640137 [ 7099 ] {6282d312-92a4-48e7-a654-17e0c458ce71} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT median(NULL, NULL, NULL);)
2024.11.04 23:20:26.715508 [ 7099 ] {6282d312-92a4-48e7-a654-17e0c458ce71} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:26.846351 [ 7099 ] {bfed44c8-04a4-498a-ab6c-c1ec03b27991} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT medianBFloat16(NULL, NULL, NULL);)
2024.11.04 23:20:26.907879 [ 7099 ] {bfed44c8-04a4-498a-ab6c-c1ec03b27991} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:27.035499 [ 3949 ] {fc4ad5f1-d917-45b5-9772-388380be751e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47214) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:27.038836 [ 3949 ] {fc4ad5f1-d917-45b5-9772-388380be751e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:27.547418 [ 7099 ] {493d9c7a-1258-44e3-ac7e-2ed9a67e48e5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT medianExact(NULL, NULL, NULL);)
2024.11.04 23:20:27.551569 [ 7099 ] {493d9c7a-1258-44e3-ac7e-2ed9a67e48e5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:27.607208 [ 7099 ] {5befc849-8277-4e25-81dc-8c18269174c0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT medianExactHigh(NULL, NULL, NULL);)
2024.11.04 23:20:27.612151 [ 7099 ] {5befc849-8277-4e25-81dc-8c18269174c0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:27.811898 [ 7099 ] {f157573b-278f-45cb-a773-844ebac4eba7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT medianExactWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:27.816292 [ 7099 ] {f157573b-278f-45cb-a773-844ebac4eba7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:27.872291 [ 7099 ] {742fd9d4-bf50-4ed1-b791-617ec4f3d0e0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT medianGK(NULL, NULL, NULL);)
2024.11.04 23:20:27.876922 [ 7099 ] {742fd9d4-bf50-4ed1-b791-617ec4f3d0e0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:28.057548 [ 7099 ] {62b6bf37-fca9-416e-9bf6-a64673325aeb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT medianInterpolatedWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:28.062999 [ 7099 ] {62b6bf37-fca9-416e-9bf6-a64673325aeb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:28.102459 [ 3949 ] {33b2effe-3b92-4e03-bd35-b26510c01219} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47226) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:28.154036 [ 3949 ] {33b2effe-3b92-4e03-bd35-b26510c01219} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:28.631840 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 32 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:28.723732 [ 7099 ] {b66a0e33-4aeb-42bf-81e2-274179051674} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT medianTimingWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:28.727844 [ 7099 ] {b66a0e33-4aeb-42bf-81e2-274179051674} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:28.868561 [ 7099 ] {afeda529-b3d4-44f6-95ad-28a4ad2d8353} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT min(NULL, NULL, NULL);)
2024.11.04 23:20:28.908397 [ 7099 ] {afeda529-b3d4-44f6-95ad-28a4ad2d8353} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:28.937164 [ 7099 ] {83d7295e-7dc5-43f0-9394-17fdd21e4581} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function min2 doesn't match: passed 3, should be 2: In scope SELECT min2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT min2(NULL, NULL, NULL);)
2024.11.04 23:20:28.939051 [ 7099 ] {83d7295e-7dc5-43f0-9394-17fdd21e4581} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function min2 doesn't match: passed 3, should be 2: In scope SELECT min2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:28.993641 [ 7099 ] {5da0dd6e-bff2-4393-ad3b-b67f27da70e9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=36, jobs=36). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT minMappedArrays(NULL, NULL, NULL);)
2024.11.04 23:20:28.997924 [ 7099 ] {5da0dd6e-bff2-4393-ad3b-b67f27da70e9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=36, jobs=36). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:29.026486 [ 7099 ] {3d4f23bc-98bf-48c9-bca6-ccc0f9b81abb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinous(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT minSampleSizeContinous(NULL, NULL, NULL);)
2024.11.04 23:20:29.028866 [ 7099 ] {3d4f23bc-98bf-48c9-bca6-ccc0f9b81abb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinous(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.079818 [ 7099 ] {1b6dc60a-2b62-434f-b04a-aa4ab8e388c7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinuous(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT minSampleSizeContinuous(NULL, NULL, NULL);)
2024.11.04 23:20:29.082545 [ 7099 ] {1b6dc60a-2b62-434f-b04a-aa4ab8e388c7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinuous(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.111632 [ 7099 ] {228ea2ae-0d2b-4354-9f86-be75880358b2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function minSampleSizeConversion doesn't match: passed 3, should be 4: In scope SELECT minSampleSizeConversion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT minSampleSizeConversion(NULL, NULL, NULL);)
2024.11.04 23:20:29.113713 [ 7099 ] {228ea2ae-0d2b-4354-9f86-be75880358b2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function minSampleSizeConversion doesn't match: passed 3, should be 4: In scope SELECT minSampleSizeConversion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.142021 [ 7099 ] {ce79d9fc-8b63-4d16-a4fb-aa3784c9d7e7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function minus doesn't match: passed 3, should be 2: In scope SELECT minus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT minus(NULL, NULL, NULL);)
2024.11.04 23:20:29.144533 [ 7099 ] {ce79d9fc-8b63-4d16-a4fb-aa3784c9d7e7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function minus doesn't match: passed 3, should be 2: In scope SELECT minus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.260588 [ 7099 ] {0dd1097e-48ee-4f68-8cef-2dc05eac6c84} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT mismatches(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mismatches(NULL, NULL, NULL);)
2024.11.04 23:20:29.262436 [ 7099 ] {0dd1097e-48ee-4f68-8cef-2dc05eac6c84} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT mismatches(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.289669 [ 7099 ] {2fe11a52-3676-4a46-823a-3aa7b117a3d3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT mod(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mod(NULL, NULL, NULL);)
2024.11.04 23:20:29.289840 [ 3949 ] {839c19e1-89ea-4cfc-9889-a4a7ac0ff9a4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47234) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:29.292196 [ 7099 ] {2fe11a52-3676-4a46-823a-3aa7b117a3d3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT mod(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.350144 [ 3949 ] {839c19e1-89ea-4cfc-9889-a4a7ac0ff9a4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:29.373320 [ 7099 ] {8f84fe74-3ff0-4a8f-a375-a2b9ad5e7493} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT modulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT modulo(NULL, NULL, NULL);)
2024.11.04 23:20:29.375681 [ 7099 ] {8f84fe74-3ff0-4a8f-a375-a2b9ad5e7493} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT modulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.399381 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_114_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:29.404644 [ 7099 ] {59d2f9fb-64e5-45b2-8c2e-f6e5b4b4f187} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function moduloLegacy doesn't match: passed 3, should be 2: In scope SELECT moduloLegacy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT moduloLegacy(NULL, NULL, NULL);)
2024.11.04 23:20:29.406607 [ 7099 ] {59d2f9fb-64e5-45b2-8c2e-f6e5b4b4f187} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function moduloLegacy doesn't match: passed 3, should be 2: In scope SELECT moduloLegacy(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.480795 [ 7099 ] {aca2c857-1167-48cb-877b-a2727c515874} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function moduloOrZero doesn't match: passed 3, should be 2: In scope SELECT moduloOrZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT moduloOrZero(NULL, NULL, NULL);)
2024.11.04 23:20:29.482829 [ 7099 ] {aca2c857-1167-48cb-877b-a2727c515874} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function moduloOrZero doesn't match: passed 3, should be 2: In scope SELECT moduloOrZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.510773 [ 7099 ] {6d69587c-ab51-485e-8f05-76352d84b13c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function monthName doesn't match: passed 3, should be 1: In scope SELECT monthName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT monthName(NULL, NULL, NULL);)
2024.11.04 23:20:29.512776 [ 7099 ] {6d69587c-ab51-485e-8f05-76352d84b13c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function monthName doesn't match: passed 3, should be 1: In scope SELECT monthName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.541620 [ 7099 ] {841cb3fa-6fd8-496b-a853-a36ecf6f50ed} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function mortonDecode doesn't match: passed 3, should be 2: In scope SELECT mortonDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT mortonDecode(NULL, NULL, NULL);)
2024.11.04 23:20:29.543675 [ 7099 ] {841cb3fa-6fd8-496b-a853-a36ecf6f50ed} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function mortonDecode doesn't match: passed 3, should be 2: In scope SELECT mortonDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:29.981740 [ 7099 ] {b34e551f-062f-4c0b-abf9-a476d08beb0d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiMatchAllIndices doesn't match: passed 3, should be 2: In scope SELECT multiMatchAllIndices(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiMatchAllIndices(NULL, NULL, NULL);)
2024.11.04 23:20:29.983943 [ 7099 ] {b34e551f-062f-4c0b-abf9-a476d08beb0d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiMatchAllIndices doesn't match: passed 3, should be 2: In scope SELECT multiMatchAllIndices(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.013671 [ 7099 ] {15923af7-7646-41e5-a77f-2a32c6c745d1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiMatchAny doesn't match: passed 3, should be 2: In scope SELECT multiMatchAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiMatchAny(NULL, NULL, NULL);)
2024.11.04 23:20:30.015613 [ 7099 ] {15923af7-7646-41e5-a77f-2a32c6c745d1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiMatchAny doesn't match: passed 3, should be 2: In scope SELECT multiMatchAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.045016 [ 7099 ] {875af322-0187-4a9f-b851-a371010f579c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiMatchAnyIndex doesn't match: passed 3, should be 2: In scope SELECT multiMatchAnyIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiMatchAnyIndex(NULL, NULL, NULL);)
2024.11.04 23:20:30.047359 [ 7099 ] {875af322-0187-4a9f-b851-a371010f579c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiMatchAnyIndex doesn't match: passed 3, should be 2: In scope SELECT multiMatchAnyIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.076376 [ 7099 ] {72812207-e432-4783-8220-eb410bf86ebb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositions doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositions(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAllPositions(NULL, NULL, NULL);)
2024.11.04 23:20:30.079104 [ 7099 ] {72812207-e432-4783-8220-eb410bf86ebb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositions doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositions(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.115294 [ 7099 ] {ead40a45-80fa-4ac8-8fab-dce3c49a0098} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAllPositionsCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:30.117724 [ 7099 ] {ead40a45-80fa-4ac8-8fab-dce3c49a0098} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.161006 [ 3539 ] {} <Warning> DNSResolver: Cannot resolve host (localhosy), error 0: Host not found.
2024.11.04 23:20:30.270414 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_105_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:30.275151 [ 7099 ] {1720c64e-26de-46cf-9bf5-5ccf6d419302} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAllPositionsCaseInsensitiveUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.278138 [ 7099 ] {1720c64e-26de-46cf-9bf5-5ccf6d419302} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.307130 [ 7099 ] {bc8c0dab-b43d-4c68-86d3-1f5e19a15b42} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAllPositionsUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.309063 [ 7099 ] {bc8c0dab-b43d-4c68-86d3-1f5e19a15b42} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.337053 [ 7099 ] {ed4a1d06-204f-479b-95dd-47e013711653} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAny doesn't match: passed 3, should be 2: In scope SELECT multiSearchAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAny(NULL, NULL, NULL);)
2024.11.04 23:20:30.338862 [ 7099 ] {ed4a1d06-204f-479b-95dd-47e013711653} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAny doesn't match: passed 3, should be 2: In scope SELECT multiSearchAny(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.367888 [ 7099 ] {7a93eca0-221d-4e10-baf9-4ae5db5d98f3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAnyCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:30.370098 [ 7099 ] {7a93eca0-221d-4e10-baf9-4ae5db5d98f3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.399519 [ 7099 ] {c8c76b6b-3f5b-4ac2-a9fb-a4a3f604e581} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAnyCaseInsensitiveUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.401641 [ 7099 ] {c8c76b6b-3f5b-4ac2-a9fb-a4a3f604e581} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.427536 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:30.429876 [ 7099 ] {04d9b873-a07b-4b11-9737-d21d960dd000} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchAnyUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.431875 [ 7099 ] {04d9b873-a07b-4b11-9737-d21d960dd000} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.457077 [ 7099 ] {bcfe8e6a-c38d-4c4f-91e3-e0fde66a9c5e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndex doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstIndex(NULL, NULL, NULL);)
2024.11.04 23:20:30.458820 [ 7099 ] {bcfe8e6a-c38d-4c4f-91e3-e0fde66a9c5e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndex doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.483896 [ 7099 ] {db066fa3-d4ba-4de3-8fbc-d4a2e6497483} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstIndexCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:30.486477 [ 7099 ] {db066fa3-d4ba-4de3-8fbc-d4a2e6497483} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.512373 [ 7099 ] {38e4fbcd-dd42-498d-9abb-abdcc944e6f6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstIndexCaseInsensitiveUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.514358 [ 7099 ] {38e4fbcd-dd42-498d-9abb-abdcc944e6f6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.539728 [ 7099 ] {6c84819d-88c9-42f9-a27d-6680954a9a4f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstIndexUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.623138 [ 7099 ] {6c84819d-88c9-42f9-a27d-6680954a9a4f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.650237 [ 7099 ] {f2bb4175-7c06-4e95-9006-610ffcbaa631} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPosition doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPosition(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstPosition(NULL, NULL, NULL);)
2024.11.04 23:20:30.652503 [ 7099 ] {f2bb4175-7c06-4e95-9006-610ffcbaa631} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPosition doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPosition(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.679996 [ 7099 ] {dc469287-dca1-4338-9f6a-b06e15a04c90} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstPositionCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:30.682414 [ 7099 ] {dc469287-dca1-4338-9f6a-b06e15a04c90} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.708206 [ 7099 ] {b3690439-b046-4fb2-9316-348ecf6b79cd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstPositionCaseInsensitiveUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.768146 [ 7099 ] {b3690439-b046-4fb2-9316-348ecf6b79cd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.794753 [ 7099 ] {02dc2dad-af39-4991-9db8-cef8fd4f611a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiSearchFirstPositionUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:30.797034 [ 7099 ] {02dc2dad-af39-4991-9db8-cef8fd4f611a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:30.826421 [ 7099 ] {01530e79-90ef-468c-9da9-ca8d15320a1f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function multiply doesn't match: passed 3, should be 2: In scope SELECT multiply(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT multiply(NULL, NULL, NULL);)
2024.11.04 23:20:30.838937 [ 7099 ] {01530e79-90ef-468c-9da9-ca8d15320a1f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiply doesn't match: passed 3, should be 2: In scope SELECT multiply(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:31.313294 [ 7099 ] {4b9f97c5-c862-4906-b7d9-0980d4e15357} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function negate doesn't match: passed 3, should be 1: In scope SELECT negate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT negate(NULL, NULL, NULL);)
2024.11.04 23:20:31.315260 [ 7099 ] {4b9f97c5-c862-4906-b7d9-0980d4e15357} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function negate doesn't match: passed 3, should be 1: In scope SELECT negate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:31.341502 [ 7099 ] {502ed90d-03a0-4dea-859f-46ea6d22cd3c} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of second argument of function neighbor - should be an integer: In scope SELECT neighbor(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT neighbor(NULL, NULL, NULL);)
2024.11.04 23:20:31.343631 [ 7099 ] {502ed90d-03a0-4dea-859f-46ea6d22cd3c} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of second argument of function neighbor - should be an integer: In scope SELECT neighbor(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:31.411328 [ 7099 ] {a4e6425d-52c3-4804-a5b1-fb5ad7e77aaf} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function netloc doesn't match: passed 3, should be 1: In scope SELECT netloc(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT netloc(NULL, NULL, NULL);)
2024.11.04 23:20:31.413606 [ 7099 ] {a4e6425d-52c3-4804-a5b1-fb5ad7e77aaf} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function netloc doesn't match: passed 3, should be 1: In scope SELECT netloc(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:31.440735 [ 7099 ] {b6c54bdf-1a19-47d7-b403-2ea2886ed0c5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistance doesn't match: passed 3, should be 2: In scope SELECT ngramDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramDistance(NULL, NULL, NULL);)
2024.11.04 23:20:31.443075 [ 7099 ] {b6c54bdf-1a19-47d7-b403-2ea2886ed0c5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistance doesn't match: passed 3, should be 2: In scope SELECT ngramDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:31.470268 [ 7099 ] {70163e8c-b8f3-496c-b2c4-987fff61f675} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramDistanceCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:31.472620 [ 7099 ] {70163e8c-b8f3-496c-b2c4-987fff61f675} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:31.500199 [ 7099 ] {03a78e1b-ed9e-478d-a19a-db7da8ffdd2d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramDistanceCaseInsensitiveUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:31.502648 [ 7099 ] {03a78e1b-ed9e-478d-a19a-db7da8ffdd2d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:31.533015 [ 7099 ] {a8a84969-c53b-41ed-8d75-d7c277a321c0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramDistanceUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:31.535330 [ 7099 ] {a8a84969-c53b-41ed-8d75-d7c277a321c0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:31.959613 [ 3739 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:31.959731 [ 3739 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:32.054457 [ 7099 ] {0465f934-d20e-463c-ad10-42def56b3a9b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearch doesn't match: passed 3, should be 2: In scope SELECT ngramSearch(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramSearch(NULL, NULL, NULL);)
2024.11.04 23:20:32.056613 [ 7099 ] {0465f934-d20e-463c-ad10-42def56b3a9b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearch doesn't match: passed 3, should be 2: In scope SELECT ngramSearch(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.081576 [ 7099 ] {e493c69b-b6d5-4462-b4f3-7d6e6bed8575} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramSearchCaseInsensitive(NULL, NULL, NULL);)
2024.11.04 23:20:32.083624 [ 7099 ] {e493c69b-b6d5-4462-b4f3-7d6e6bed8575} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitive(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.128396 [ 7099 ] {131404b6-e5d7-4573-a36b-7a4fe5d70bb8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramSearchCaseInsensitiveUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:32.130524 [ 7099 ] {131404b6-e5d7-4573-a36b-7a4fe5d70bb8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitiveUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.155587 [ 7099 ] {04177405-8cbd-45eb-b4f5-989c8b7b3653} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearchUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngramSearchUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:32.157697 [ 7099 ] {04177405-8cbd-45eb-b4f5-989c8b7b3653} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearchUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.244021 [ 4802 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:32.244939 [ 3456 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.245057 [ 3479 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.245156 [ 3504 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.245264 [ 3509 ] {} <Warning> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.245386 [ 3496 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.245453 [ 3525 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.245608 [ 3491 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.245735 [ 3573 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:32.246486 [ 3494 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:32.246540 [ 3584 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:32.247106 [ 3511 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:32.521414 [ 7099 ] {4adb02e8-2751-48fd-95db-4b9b1a353381} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function ngrams doesn't match: passed 3, should be 2: In scope SELECT ngrams(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ngrams(NULL, NULL, NULL);)
2024.11.04 23:20:32.523844 [ 7099 ] {4adb02e8-2751-48fd-95db-4b9b1a353381} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngrams doesn't match: passed 3, should be 2: In scope SELECT ngrams(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.641899 [ 7099 ] {82cdf02e-72e5-4cff-a91a-65af3c41903b} <Error> executeQuery: Code: 36. DB::Exception: Argument 0 must be a number, 'Nullable(Nothing)' given. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT nonNegativeDerivative(NULL, NULL, NULL);)
2024.11.04 23:20:32.643776 [ 7099 ] {82cdf02e-72e5-4cff-a91a-65af3c41903b} <Error> TCPHandler: Code: 36. DB::Exception: Argument 0 must be a number, 'Nullable(Nothing)' given. (BAD_ARGUMENTS)
2024.11.04 23:20:32.673572 [ 7099 ] {7ffaeced-bcf6-48bf-aef9-4ef6b88fe3c5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT normL1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normL1(NULL, NULL, NULL);)
2024.11.04 23:20:32.675686 [ 7099 ] {7ffaeced-bcf6-48bf-aef9-4ef6b88fe3c5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT normL1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.706519 [ 7099 ] {4c473b3b-b80a-41f9-bdab-db3ffeac4899} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT normL2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normL2(NULL, NULL, NULL);)
2024.11.04 23:20:32.709020 [ 7099 ] {4c473b3b-b80a-41f9-bdab-db3ffeac4899} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT normL2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.737583 [ 7099 ] {0aba951a-b273-4d91-ad30-a4dbaef7cabc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT normL2Squared(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normL2Squared(NULL, NULL, NULL);)
2024.11.04 23:20:32.739594 [ 7099 ] {0aba951a-b273-4d91-ad30-a4dbaef7cabc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT normL2Squared(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.766034 [ 7099 ] {e61563e6-a36e-40c1-baad-7cc3921971b6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT normLinf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normLinf(NULL, NULL, NULL);)
2024.11.04 23:20:32.767814 [ 7099 ] {e61563e6-a36e-40c1-baad-7cc3921971b6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT normLinf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.790984 [ 7099 ] {0e4cc151-2b58-4f18-aec8-3f114c4492be} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT normLp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normLp(NULL, NULL, NULL);)
2024.11.04 23:20:32.792909 [ 7099 ] {0e4cc151-2b58-4f18-aec8-3f114c4492be} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT normLp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.888561 [ 7099 ] {efa9faea-6f79-4f75-a505-4200465d65f5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeL1(NULL, NULL, NULL);)
2024.11.04 23:20:32.890669 [ 7099 ] {efa9faea-6f79-4f75-a505-4200465d65f5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL1(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.917851 [ 7099 ] {f8731683-c297-4e2d-8541-ac01a0947c95} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeL2(NULL, NULL, NULL);)
2024.11.04 23:20:32.919835 [ 7099 ] {f8731683-c297-4e2d-8541-ac01a0947c95} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.946658 [ 7099 ] {36cc7743-e8c3-4449-8908-71813bee1db3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT normalizeLinf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeLinf(NULL, NULL, NULL);)
2024.11.04 23:20:32.948844 [ 7099 ] {36cc7743-e8c3-4449-8908-71813bee1db3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT normalizeLinf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:32.976634 [ 7099 ] {adefcefa-6b1e-4a77-80e4-76b0643ce184} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT normalizeLp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeLp(NULL, NULL, NULL);)
2024.11.04 23:20:32.978911 [ 7099 ] {adefcefa-6b1e-4a77-80e4-76b0643ce184} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT normalizeLp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.005101 [ 7099 ] {54ff2d22-1bae-40b9-8026-23aab8ca352d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeQuery doesn't match: passed 3, should be 1: In scope SELECT normalizeQuery(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeQuery(NULL, NULL, NULL);)
2024.11.04 23:20:33.007215 [ 7099 ] {54ff2d22-1bae-40b9-8026-23aab8ca352d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeQuery doesn't match: passed 3, should be 1: In scope SELECT normalizeQuery(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.033461 [ 7099 ] {c37e88b9-52d3-41db-bdf4-e7da7a00327c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeQueryKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizeQueryKeepNames(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeQueryKeepNames(NULL, NULL, NULL);)
2024.11.04 23:20:33.035736 [ 7099 ] {c37e88b9-52d3-41db-bdf4-e7da7a00327c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeQueryKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizeQueryKeepNames(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.098217 [ 7099 ] {35d2f143-e89b-4f40-b0db-f60a020460d3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeUTF8NFC(NULL, NULL, NULL);)
2024.11.04 23:20:33.100173 [ 7099 ] {35d2f143-e89b-4f40-b0db-f60a020460d3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.126568 [ 7099 ] {380d8c42-5593-4d4d-9188-e78198c9df64} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFD(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeUTF8NFD(NULL, NULL, NULL);)
2024.11.04 23:20:33.198877 [ 7099 ] {380d8c42-5593-4d4d-9188-e78198c9df64} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFD(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.227529 [ 7099 ] {a6827119-fc78-4800-ba0c-8ded9a16c6a2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeUTF8NFKC(NULL, NULL, NULL);)
2024.11.04 23:20:33.230032 [ 7099 ] {a6827119-fc78-4800-ba0c-8ded9a16c6a2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.256732 [ 7099 ] {f0e8273f-4530-41d6-bfb5-b2d29028113c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKD(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizeUTF8NFKD(NULL, NULL, NULL);)
2024.11.04 23:20:33.258612 [ 7099 ] {f0e8273f-4530-41d6-bfb5-b2d29028113c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKD(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.334498 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ikb/ijxlpcljgmaaclqynjfcuxhbwofii, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 14cef6c1-fce4-4640-83ec-a730752e186c, upload has finished false.
2024.11.04 23:20:33.334741 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ikb/ijxlpcljgmaaclqynjfcuxhbwofii, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 14cef6c1-fce4-4640-83ec-a730752e186c, upload has finished false
2024.11.04 23:20:33.346172 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ikb/ijxlpcljgmaaclqynjfcuxhbwofii, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 14cef6c1-fce4-4640-83ec-a730752e186c, upload has finished false
2024.11.04 23:20:33.350926 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 52 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:33.378342 [ 7099 ] {57309e8c-4689-490f-aab5-00e9f6b20615} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHash doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizedQueryHash(NULL, NULL, NULL);)
2024.11.04 23:20:33.473277 [ 7099 ] {57309e8c-4689-490f-aab5-00e9f6b20615} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHash doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.500857 [ 7099 ] {c12db8b8-22f1-41a2-a451-4cd34193ccec} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHashKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHashKeepNames(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT normalizedQueryHashKeepNames(NULL, NULL, NULL);)
2024.11.04 23:20:33.503012 [ 7099 ] {c12db8b8-22f1-41a2-a451-4cd34193ccec} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHashKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHashKeepNames(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.529572 [ 7099 ] {b676c3d2-6495-4e1b-b2ad-dcd000a83149} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function not doesn't match: passed 3, should be 1: In scope SELECT not(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT not(NULL, NULL, NULL);)
2024.11.04 23:20:33.531557 [ 7099 ] {b676c3d2-6495-4e1b-b2ad-dcd000a83149} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function not doesn't match: passed 3, should be 1: In scope SELECT not(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.604957 [ 7099 ] {9c284c7a-4180-4d80-8ea7-f42881de752e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function notEmpty doesn't match: passed 3, should be 1: In scope SELECT notEmpty(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notEmpty(NULL, NULL, NULL);)
2024.11.04 23:20:33.606426 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:33.607080 [ 7099 ] {9c284c7a-4180-4d80-8ea7-f42881de752e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function notEmpty doesn't match: passed 3, should be 1: In scope SELECT notEmpty(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.634024 [ 7099 ] {9e8839aa-1464-4db8-a9a5-455d8540c8cb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function notEquals doesn't match: passed 3, should be 2: In scope SELECT notEquals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notEquals(NULL, NULL, NULL);)
2024.11.04 23:20:33.635932 [ 7099 ] {9e8839aa-1464-4db8-a9a5-455d8540c8cb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function notEquals doesn't match: passed 3, should be 2: In scope SELECT notEquals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.664023 [ 7099 ] {d81e5f03-552a-4976-8795-32c5c88fcba7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function notILike doesn't match: passed 3, should be 2: In scope SELECT notILike(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notILike(NULL, NULL, NULL);)
2024.11.04 23:20:33.666060 [ 7099 ] {d81e5f03-552a-4976-8795-32c5c88fcba7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function notILike doesn't match: passed 3, should be 2: In scope SELECT notILike(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.689654 [ 7099 ] {3901aa65-8fb7-454b-b101-10bc1533037b} <Error> executeQuery: Code: 42. DB::Exception: Function 'notIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notIn(NULL, NULL, NULL);)
2024.11.04 23:20:33.691408 [ 7099 ] {3901aa65-8fb7-454b-b101-10bc1533037b} <Error> TCPHandler: Code: 42. DB::Exception: Function 'notIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.714501 [ 7099 ] {12e6e73a-0142-40d1-9afa-b38fda182221} <Error> executeQuery: Code: 42. DB::Exception: Function 'notInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notInIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:33.716742 [ 7099 ] {12e6e73a-0142-40d1-9afa-b38fda182221} <Error> TCPHandler: Code: 42. DB::Exception: Function 'notInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.741184 [ 7099 ] {ca6e102b-8171-43d7-915b-c5396e734d0d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function notLike doesn't match: passed 3, should be 2: In scope SELECT notLike(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notLike(NULL, NULL, NULL);)
2024.11.04 23:20:33.743227 [ 7099 ] {ca6e102b-8171-43d7-915b-c5396e734d0d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function notLike doesn't match: passed 3, should be 2: In scope SELECT notLike(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.818781 [ 7099 ] {7100e34a-1fa9-42ab-8096-e48f459c2f18} <Error> executeQuery: Code: 42. DB::Exception: Function 'notNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notNullIn(NULL, NULL, NULL);)
2024.11.04 23:20:33.821071 [ 7099 ] {7100e34a-1fa9-42ab-8096-e48f459c2f18} <Error> TCPHandler: Code: 42. DB::Exception: Function 'notNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:33.846816 [ 7099 ] {1611410b-d12c-485c-94ea-f2d524df5854} <Error> executeQuery: Code: 42. DB::Exception: Function 'notNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT notNullInIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:33.852407 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_115_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:33.940460 [ 7099 ] {1611410b-d12c-485c-94ea-f2d524df5854} <Error> TCPHandler: Code: 42. DB::Exception: Function 'notNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:34.340626 [ 7099 ] {f2a7c083-dc0b-4997-835c-172aa8262f7b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT nothingUInt64(NULL, NULL, NULL);)
2024.11.04 23:20:34.345017 [ 7099 ] {f2a7c083-dc0b-4997-835c-172aa8262f7b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:34.375232 [ 7099 ] {ccd08321-d0aa-4bde-9bff-1e7ea87909a5} <Error> executeQuery: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT now(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT now(NULL, NULL, NULL);)
2024.11.04 23:20:34.377725 [ 7099 ] {ccd08321-d0aa-4bde-9bff-1e7ea87909a5} <Error> TCPHandler: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT now(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:34.544922 [ 7099 ] {60f6a658-3675-4ada-b813-642d6d4f3281} <Error> executeQuery: Code: 36. DB::Exception: Function nth_value takes exactly two arguments. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT nth_value(NULL, NULL, NULL);)
2024.11.04 23:20:34.546885 [ 7099 ] {60f6a658-3675-4ada-b813-642d6d4f3281} <Error> TCPHandler: Code: 36. DB::Exception: Function nth_value takes exactly two arguments. (BAD_ARGUMENTS)
2024.11.04 23:20:34.657562 [ 7099 ] {142837ac-27b5-4706-ae32-41b4665a191e} <Error> executeQuery: Code: 36. DB::Exception: Function ntile takes exactly one argument. (BAD_ARGUMENTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ntile(NULL, NULL, NULL);)
2024.11.04 23:20:34.659794 [ 7099 ] {142837ac-27b5-4706-ae32-41b4665a191e} <Error> TCPHandler: Code: 36. DB::Exception: Function ntile takes exactly one argument. (BAD_ARGUMENTS)
2024.11.04 23:20:34.687976 [ 7099 ] {32b6ad17-4612-41bb-ad14-c9e52286a505} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function nullIf doesn't match: passed 3, should be 2: In scope SELECT nullIf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT nullIf(NULL, NULL, NULL);)
2024.11.04 23:20:34.690210 [ 7099 ] {32b6ad17-4612-41bb-ad14-c9e52286a505} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function nullIf doesn't match: passed 3, should be 2: In scope SELECT nullIf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:34.717191 [ 7099 ] {40128400-c459-4996-864f-3a48b627cee8} <Error> executeQuery: Code: 42. DB::Exception: Function 'nullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT nullIn(NULL, NULL, NULL);)
2024.11.04 23:20:34.719341 [ 7099 ] {40128400-c459-4996-864f-3a48b627cee8} <Error> TCPHandler: Code: 42. DB::Exception: Function 'nullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:34.746186 [ 7099 ] {d019c6bf-1f8b-468c-bee2-e93e0a28f6b0} <Error> executeQuery: Code: 42. DB::Exception: Function 'nullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT nullInIgnoreSet(NULL, NULL, NULL);)
2024.11.04 23:20:34.748384 [ 7099 ] {d019c6bf-1f8b-468c-bee2-e93e0a28f6b0} <Error> TCPHandler: Code: 42. DB::Exception: Function 'nullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:35.595390 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:36.162119 [ 2935 ] {a8b13c4b-98b8-422b-ab4f-a71b151e4ac2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50962) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:36.165182 [ 2935 ] {a8b13c4b-98b8-422b-ab4f-a71b151e4ac2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:36.328547 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:36.349795 [ 7099 ] {7a2aa5d1-4c21-473a-b4b0-1362d8188814} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function path doesn't match: passed 3, should be 1: In scope SELECT path(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT path(NULL, NULL, NULL);)
2024.11.04 23:20:36.351916 [ 7099 ] {7a2aa5d1-4c21-473a-b4b0-1362d8188814} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function path doesn't match: passed 3, should be 1: In scope SELECT path(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.376591 [ 7099 ] {e636fc1e-c6f7-4b99-b89b-c0143e0195cd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function pathFull doesn't match: passed 3, should be 1: In scope SELECT pathFull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT pathFull(NULL, NULL, NULL);)
2024.11.04 23:20:36.379016 [ 7099 ] {e636fc1e-c6f7-4b99-b89b-c0143e0195cd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function pathFull doesn't match: passed 3, should be 1: In scope SELECT pathFull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.403322 [ 7099 ] {bc06988c-0177-461e-bbbf-39e3bb99a6f6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function pi doesn't match: passed 3, should be 0: In scope SELECT pi(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT pi(NULL, NULL, NULL);)
2024.11.04 23:20:36.405483 [ 7099 ] {bc06988c-0177-461e-bbbf-39e3bb99a6f6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function pi doesn't match: passed 3, should be 0: In scope SELECT pi(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.429244 [ 7099 ] {c3e614d4-50de-43be-8922-380ec04448af} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function plus doesn't match: passed 3, should be 2: In scope SELECT plus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT plus(NULL, NULL, NULL);)
2024.11.04 23:20:36.431133 [ 7099 ] {c3e614d4-50de-43be-8922-380ec04448af} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function plus doesn't match: passed 3, should be 2: In scope SELECT plus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.454179 [ 7099 ] {8c25b999-4a4a-47ca-a953-6f36352d4073} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT pmod(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT pmod(NULL, NULL, NULL);)
2024.11.04 23:20:36.456050 [ 7099 ] {8c25b999-4a4a-47ca-a953-6f36352d4073} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT pmod(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.659947 [ 7099 ] {b003ac1d-2f78-46a3-af70-bc3d81a87bd1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonAreaCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonAreaCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonAreaCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:36.661970 [ 7099 ] {b003ac1d-2f78-46a3-af70-bc3d81a87bd1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonAreaCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonAreaCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.686197 [ 7099 ] {c54106f1-06c7-47f4-bf78-2c3249778c4a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonAreaSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonAreaSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonAreaSpherical(NULL, NULL, NULL);)
2024.11.04 23:20:36.688270 [ 7099 ] {c54106f1-06c7-47f4-bf78-2c3249778c4a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonAreaSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonAreaSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.711934 [ 7099 ] {62cf8257-c23a-4b42-889b-a0af5d757082} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonConvexHullCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonConvexHullCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonConvexHullCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:36.714198 [ 7099 ] {62cf8257-c23a-4b42-889b-a0af5d757082} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonConvexHullCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonConvexHullCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.738101 [ 7099 ] {f5bc9465-8018-43f8-a708-058f7a95ddd8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonPerimeterCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:36.740228 [ 7099 ] {f5bc9465-8018-43f8-a708-058f7a95ddd8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.764506 [ 7099 ] {3ff96978-f7ea-42e5-9823-60405289db83} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonPerimeterSpherical(NULL, NULL, NULL);)
2024.11.04 23:20:36.766315 [ 7099 ] {3ff96978-f7ea-42e5-9823-60405289db83} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.815281 [ 7099 ] {dca3b539-3fc2-4bcd-8dfc-b70ee324603b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsDistanceCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:36.817215 [ 7099 ] {dca3b539-3fc2-4bcd-8dfc-b70ee324603b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.841437 [ 7099 ] {c7bed500-e487-46c9-955e-737578050285} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsDistanceSpherical(NULL, NULL, NULL);)
2024.11.04 23:20:36.843304 [ 7099 ] {c7bed500-e487-46c9-955e-737578050285} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.866286 [ 7099 ] {baee4478-ed2c-4fb6-9644-23eb7654f0e0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsEqualsCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsEqualsCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsEqualsCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:36.868171 [ 7099 ] {baee4478-ed2c-4fb6-9644-23eb7654f0e0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsEqualsCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsEqualsCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.891175 [ 7099 ] {c1a57aa6-2666-4439-b67f-5f686b24458e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsIntersectionCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:36.893184 [ 7099 ] {c1a57aa6-2666-4439-b67f-5f686b24458e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.929431 [ 7099 ] {f0b4ec53-951f-448e-8d49-e7aa6e822af3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsIntersectionSpherical(NULL, NULL, NULL);)
2024.11.04 23:20:36.931496 [ 7099 ] {f0b4ec53-951f-448e-8d49-e7aa6e822af3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.957920 [ 7099 ] {996f70c3-4372-4ebb-bceb-9f73ac13815e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsSymDifferenceCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:36.959739 [ 7099 ] {996f70c3-4372-4ebb-bceb-9f73ac13815e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:36.984861 [ 7099 ] {7e41c6bb-ca3a-4eb9-b085-bfc2106c44c7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsSymDifferenceSpherical(NULL, NULL, NULL);)
2024.11.04 23:20:36.986697 [ 7099 ] {7e41c6bb-ca3a-4eb9-b085-bfc2106c44c7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.008411 [ 7099 ] {573012a8-b1d1-4de8-9318-3349231355ec} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsUnionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsUnionCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:37.010222 [ 7099 ] {573012a8-b1d1-4de8-9318-3349231355ec} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsUnionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.036192 [ 7099 ] {084bda0e-81e3-443a-bda1-bd46a34714b7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsUnionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsUnionSpherical(NULL, NULL, NULL);)
2024.11.04 23:20:37.038392 [ 7099 ] {084bda0e-81e3-443a-bda1-bd46a34714b7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsUnionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.072429 [ 7099 ] {841de25b-5a9f-41a1-a4c4-c757cf18f753} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsWithinCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsWithinCartesian(NULL, NULL, NULL);)
2024.11.04 23:20:37.074498 [ 7099 ] {841de25b-5a9f-41a1-a4c4-c757cf18f753} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsWithinCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinCartesian(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.125802 [ 7099 ] {4309552c-4f60-4853-af2e-0f92b8598935} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsWithinSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT polygonsWithinSpherical(NULL, NULL, NULL);)
2024.11.04 23:20:37.128111 [ 7099 ] {4309552c-4f60-4853-af2e-0f92b8598935} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsWithinSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinSpherical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.247990 [ 2935 ] {dd41138b-67ff-4152-8b06-b84ca94c1e35} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50966) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:37.251576 [ 2935 ] {dd41138b-67ff-4152-8b06-b84ca94c1e35} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:37.337498 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_115_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:37.474923 [ 7099 ] {e06ee0b9-e3db-4993-b10f-d7b8f37f7a58} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positiveModulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT positiveModulo(NULL, NULL, NULL);)
2024.11.04 23:20:37.476768 [ 7099 ] {e06ee0b9-e3db-4993-b10f-d7b8f37f7a58} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positiveModulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.501056 [ 7099 ] {57c8cec4-9116-46b7-9cc4-ae044c17f7fb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positive_modulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT positive_modulo(NULL, NULL, NULL);)
2024.11.04 23:20:37.502966 [ 7099 ] {57c8cec4-9116-46b7-9cc4-ae044c17f7fb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positive_modulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.535748 [ 7099 ] {fbf1537e-6b5b-4c20-addb-830259ba47ad} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT pow(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT pow(NULL, NULL, NULL);)
2024.11.04 23:20:37.537776 [ 7099 ] {fbf1537e-6b5b-4c20-addb-830259ba47ad} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT pow(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.571910 [ 7099 ] {eae44c4f-bd30-4521-a20b-e13876f87aa2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT power(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT power(NULL, NULL, NULL);)
2024.11.04 23:20:37.573878 [ 7099 ] {eae44c4f-bd30-4521-a20b-e13876f87aa2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT power(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.657365 [ 7099 ] {f113733c-0c00-41de-8f75-7be5033888ec} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function proportionsZTest doesn't match: passed 3, should be 6: In scope SELECT proportionsZTest(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT proportionsZTest(NULL, NULL, NULL);)
2024.11.04 23:20:37.659591 [ 7099 ] {f113733c-0c00-41de-8f75-7be5033888ec} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function proportionsZTest doesn't match: passed 3, should be 6: In scope SELECT proportionsZTest(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.686568 [ 7099 ] {a7510178-07b2-416b-bd42-6c20ae1ced2d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function protocol doesn't match: passed 3, should be 1: In scope SELECT protocol(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT protocol(NULL, NULL, NULL);)
2024.11.04 23:20:37.688736 [ 7099 ] {a7510178-07b2-416b-bd42-6c20ae1ced2d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function protocol doesn't match: passed 3, should be 1: In scope SELECT protocol(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.782216 [ 7099 ] {96058224-84a5-47be-8f6e-0b363384bd16} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function punycodeDecode doesn't match: passed 3, should be 1: In scope SELECT punycodeDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT punycodeDecode(NULL, NULL, NULL);)
2024.11.04 23:20:37.784465 [ 7099 ] {96058224-84a5-47be-8f6e-0b363384bd16} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function punycodeDecode doesn't match: passed 3, should be 1: In scope SELECT punycodeDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.927817 [ 7099 ] {4364ce29-f0e0-49f6-8f11-989eabe0bd46} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function punycodeEncode doesn't match: passed 3, should be 1: In scope SELECT punycodeEncode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT punycodeEncode(NULL, NULL, NULL);)
2024.11.04 23:20:37.930077 [ 7099 ] {4364ce29-f0e0-49f6-8f11-989eabe0bd46} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function punycodeEncode doesn't match: passed 3, should be 1: In scope SELECT punycodeEncode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:37.980636 [ 7099 ] {7c5c7420-2f9b-4a97-9b8b-51c9c163f11e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantile(NULL, NULL, NULL);)
2024.11.04 23:20:37.985855 [ 7099 ] {7c5c7420-2f9b-4a97-9b8b-51c9c163f11e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:38.382180 [ 7099 ] {b65d1f35-58de-47bc-a511-44a0f0e0757a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantileExact(NULL, NULL, NULL);)
2024.11.04 23:20:38.385927 [ 7099 ] {b65d1f35-58de-47bc-a511-44a0f0e0757a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:38.623541 [ 7099 ] {2a4061e2-c172-4510-aa75-347f9dc89e3e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantileExactLow(NULL, NULL, NULL);)
2024.11.04 23:20:38.627849 [ 7099 ] {2a4061e2-c172-4510-aa75-347f9dc89e3e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:38.819212 [ 7099 ] {d3daa4c5-2d19-408f-84f9-d8c12ac56e4e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantileInterpolatedWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:38.824080 [ 7099 ] {d3daa4c5-2d19-408f-84f9-d8c12ac56e4e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:38.875946 [ 7099 ] {99b948ad-5bc8-44b5-a3d4-d14d212669fe} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantileTDigest(NULL, NULL, NULL);)
2024.11.04 23:20:38.880196 [ 7099 ] {99b948ad-5bc8-44b5-a3d4-d14d212669fe} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:39.210897 [ 7099 ] {bdad64c4-66c2-49b8-9e4c-8b2cdaf7f47f} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantiles. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantiles(NULL, NULL, NULL);)
2024.11.04 23:20:39.213028 [ 7099 ] {bdad64c4-66c2-49b8-9e4c-8b2cdaf7f47f} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantiles. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.240794 [ 7099 ] {707ceb5d-69b2-4724-a279-10c0b208934f} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesBFloat16(NULL, NULL, NULL);)
2024.11.04 23:20:39.243005 [ 7099 ] {707ceb5d-69b2-4724-a279-10c0b208934f} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.294154 [ 7099 ] {2c3b232a-7297-4401-8c05-65db3f6e8416} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesBFloat16Weighted(NULL, NULL, NULL);)
2024.11.04 23:20:39.299214 [ 7099 ] {2c3b232a-7297-4401-8c05-65db3f6e8416} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:39.325228 [ 7099 ] {df19bc62-cca0-40b8-9668-edd6b385f79b} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesDD. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesDD(NULL, NULL, NULL);)
2024.11.04 23:20:39.327398 [ 7099 ] {df19bc62-cca0-40b8-9668-edd6b385f79b} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesDD. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.354730 [ 7099 ] {fcc2b88d-85f2-4eb4-94d8-b69734bb8bde} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesDeterministic(NULL, NULL, NULL);)
2024.11.04 23:20:39.357099 [ 7099 ] {fcc2b88d-85f2-4eb4-94d8-b69734bb8bde} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.385182 [ 7099 ] {041e2535-58b1-4bd0-9019-bccaca108cbf} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExact. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesExact(NULL, NULL, NULL);)
2024.11.04 23:20:39.387475 [ 7099 ] {041e2535-58b1-4bd0-9019-bccaca108cbf} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExact. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.413226 [ 7099 ] {fb27be77-bc7a-4792-a375-c3a556888d5d} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactExclusive. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesExactExclusive(NULL, NULL, NULL);)
2024.11.04 23:20:39.415372 [ 7099 ] {fb27be77-bc7a-4792-a375-c3a556888d5d} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactExclusive. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.457705 [ 7099 ] {488d4b89-f990-49bd-85e5-aea71e8f484a} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesExactHigh(NULL, NULL, NULL);)
2024.11.04 23:20:39.459887 [ 7099 ] {488d4b89-f990-49bd-85e5-aea71e8f484a} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.486679 [ 7099 ] {951dadcc-7860-4cf8-aa02-08a501baa32e} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactInclusive. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesExactInclusive(NULL, NULL, NULL);)
2024.11.04 23:20:39.489019 [ 7099 ] {951dadcc-7860-4cf8-aa02-08a501baa32e} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactInclusive. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.515289 [ 7099 ] {4d003cc3-4b38-4a86-a8da-291f73ce7165} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactLow. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesExactLow(NULL, NULL, NULL);)
2024.11.04 23:20:39.517384 [ 7099 ] {4d003cc3-4b38-4a86-a8da-291f73ce7165} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactLow. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.542008 [ 7099 ] {c7201911-c9e2-406c-827d-5edc1da995ad} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesExactWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:39.544167 [ 7099 ] {c7201911-c9e2-406c-827d-5edc1da995ad} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.569664 [ 7099 ] {51e0f3f5-fbf8-465b-9ec0-9c9b836dff9d} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesGK. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesGK(NULL, NULL, NULL);)
2024.11.04 23:20:39.571935 [ 7099 ] {51e0f3f5-fbf8-465b-9ec0-9c9b836dff9d} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesGK. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.597096 [ 7099 ] {b89c0171-d3b7-4e95-b6fd-a0d1ad1b2830} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesInterpolatedWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:39.601495 [ 7099 ] {b89c0171-d3b7-4e95-b6fd-a0d1ad1b2830} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.626429 [ 7099 ] {1a6c30f4-ae97-4337-8e18-bc0692cb3161} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTDigest. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesTDigest(NULL, NULL, NULL);)
2024.11.04 23:20:39.628479 [ 7099 ] {1a6c30f4-ae97-4337-8e18-bc0692cb3161} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTDigest. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.652988 [ 7099 ] {9969edc7-bc44-41c8-b9ca-0330e2737ed6} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesTDigestWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:39.655148 [ 7099 ] {9969edc7-bc44-41c8-b9ca-0330e2737ed6} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.678849 [ 7099 ] {36612c51-26ab-41d4-b4a6-3a8f3548ebf5} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTiming. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesTiming(NULL, NULL, NULL);)
2024.11.04 23:20:39.680521 [ 7099 ] {36612c51-26ab-41d4-b4a6-3a8f3548ebf5} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTiming. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.757377 [ 7099 ] {d3aab8da-6445-4aea-8847-0b7a3ff258ea} <Error> executeQuery: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT quantilesTimingWeighted(NULL, NULL, NULL);)
2024.11.04 23:20:39.759246 [ 7099 ] {d3aab8da-6445-4aea-8847-0b7a3ff258ea} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type Nullable(Nothing) of argument for aggregate function quantilesTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:39.788138 [ 7099 ] {2a5302be-f543-42d1-b659-195133c8eefe} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT queryID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT queryID(NULL, NULL, NULL);)
2024.11.04 23:20:39.791122 [ 7099 ] {2a5302be-f543-42d1-b659-195133c8eefe} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT queryID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:39.826861 [ 7099 ] {6ced73f2-d9bc-45ce-888e-9a26a4053117} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function queryString doesn't match: passed 3, should be 1: In scope SELECT queryString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT queryString(NULL, NULL, NULL);)
2024.11.04 23:20:39.829839 [ 7099 ] {6ced73f2-d9bc-45ce-888e-9a26a4053117} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryString doesn't match: passed 3, should be 1: In scope SELECT queryString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:39.958200 [ 7099 ] {ef3514a2-2856-408b-8674-207f19aef55c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function queryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT queryStringAndFragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT queryStringAndFragment(NULL, NULL, NULL);)
2024.11.04 23:20:39.960505 [ 7099 ] {ef3514a2-2856-408b-8674-207f19aef55c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT queryStringAndFragment(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:39.985413 [ 7099 ] {263baed0-2b82-4ae5-b3cf-522981fc2346} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT query_id(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT query_id(NULL, NULL, NULL);)
2024.11.04 23:20:39.987724 [ 7099 ] {263baed0-2b82-4ae5-b3cf-522981fc2346} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT query_id(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.030625 [ 7099 ] {0722bd4b-f87d-4a8a-9539-6ffa2e827b2a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function radians doesn't match: passed 3, should be 1: In scope SELECT radians(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT radians(NULL, NULL, NULL);)
2024.11.04 23:20:40.033479 [ 7099 ] {0722bd4b-f87d-4a8a-9539-6ffa2e827b2a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function radians doesn't match: passed 3, should be 1: In scope SELECT radians(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.061107 [ 7099 ] {51710548-bc90-4054-8c57-22731b2fde76} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT rand(NULL, NULL, NULL);)
2024.11.04 23:20:40.063462 [ 7099 ] {51710548-bc90-4054-8c57-22731b2fde76} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.087492 [ 7099 ] {28e1c505-8722-4a7f-96c3-b9843a16a4e0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT rand32(NULL, NULL, NULL);)
2024.11.04 23:20:40.089482 [ 7099 ] {28e1c505-8722-4a7f-96c3-b9843a16a4e0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.129338 [ 7099 ] {e9fe4965-6d7e-4f3c-b7e7-66b9f566cc35} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function rand64 doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT rand64(NULL, NULL, NULL);)
2024.11.04 23:20:40.134353 [ 7099 ] {e9fe4965-6d7e-4f3c-b7e7-66b9f566cc35} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function rand64 doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.226921 [ 7099 ] {0308ec6b-aa3d-4b10-a64f-46c58801464a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function randCanonical doesn't match: passed 3, should be 0 or 1.: In scope SELECT randCanonical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT randCanonical(NULL, NULL, NULL);)
2024.11.04 23:20:40.228939 [ 7099 ] {0308ec6b-aa3d-4b10-a64f-46c58801464a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function randCanonical doesn't match: passed 3, should be 0 or 1.: In scope SELECT randCanonical(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.434262 [ 7099 ] {b49f4749-d011-4aa2-b3cb-d0ad4edd1ed4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function randConstant doesn't match: passed 3, should be 0 or 1.: In scope SELECT randConstant(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT randConstant(NULL, NULL, NULL);)
2024.11.04 23:20:40.436934 [ 7099 ] {b49f4749-d011-4aa2-b3cb-d0ad4edd1ed4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function randConstant doesn't match: passed 3, should be 0 or 1.: In scope SELECT randConstant(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.472110 [ 2935 ] {cc54c4f8-793f-4033-9320-b7e80f0105f6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50990) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:40.475257 [ 2935 ] {cc54c4f8-793f-4033-9320-b7e80f0105f6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:20:40.824617 [ 7099 ] {8876e644-f952-49c7-a9bd-b641f0e9afb0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function randomFixedString doesn't match: passed 3, should be 1: In scope SELECT randomFixedString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT randomFixedString(NULL, NULL, NULL);)
2024.11.04 23:20:40.826670 [ 7099 ] {8876e644-f952-49c7-a9bd-b641f0e9afb0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function randomFixedString doesn't match: passed 3, should be 1: In scope SELECT randomFixedString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:40.930272 [ 7099 ] {5f06a2b3-4df9-4074-9ead-3f25a28b2c21} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function randomStringUTF8 doesn't match: passed 3, should be 1: In scope SELECT randomStringUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT randomStringUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:40.932148 [ 7099 ] {5f06a2b3-4df9-4074-9ead-3f25a28b2c21} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function randomStringUTF8 doesn't match: passed 3, should be 1: In scope SELECT randomStringUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:41.013624 [ 7099 ] {46ffa918-3408-407a-bff2-649a3b22435c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT rank(NULL, NULL, NULL);)
2024.11.04 23:20:41.018372 [ 7099 ] {46ffa918-3408-407a-bff2-649a3b22435c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:41.146878 [ 7099 ] {be2203f0-935d-4f0b-bbfc-7ad3f6f1664f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTMultiPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTMultiPolygon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT readWKTMultiPolygon(NULL, NULL, NULL);)
2024.11.04 23:20:41.149374 [ 7099 ] {be2203f0-935d-4f0b-bbfc-7ad3f6f1664f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTMultiPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTMultiPolygon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:41.177652 [ 7099 ] {86b78a96-a07b-47e2-80f0-90c59b631f0d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTPoint doesn't match: passed 3, should be 1: In scope SELECT readWKTPoint(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT readWKTPoint(NULL, NULL, NULL);)
2024.11.04 23:20:41.179519 [ 7099 ] {86b78a96-a07b-47e2-80f0-90c59b631f0d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTPoint doesn't match: passed 3, should be 1: In scope SELECT readWKTPoint(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:41.202227 [ 7099 ] {47ca334c-ce16-4d04-9026-84b663cc4872} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTPolygon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT readWKTPolygon(NULL, NULL, NULL);)
2024.11.04 23:20:41.204183 [ 7099 ] {47ca334c-ce16-4d04-9026-84b663cc4872} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTPolygon(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:41.227415 [ 7099 ] {8bd89c87-3d71-4352-98ee-3e2783eabd9b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTRing doesn't match: passed 3, should be 1: In scope SELECT readWKTRing(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT readWKTRing(NULL, NULL, NULL);)
2024.11.04 23:20:41.229580 [ 7099 ] {8bd89c87-3d71-4352-98ee-3e2783eabd9b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTRing doesn't match: passed 3, should be 1: In scope SELECT readWKTRing(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:41.279142 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/zwu/dxxyngidstbguliqasdynisqpyvgs, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2c6d4393-88fb-446d-98e3-77646fcb2ef8, upload has finished false.
2024.11.04 23:20:41.279352 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/zwu/dxxyngidstbguliqasdynisqpyvgs, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2c6d4393-88fb-446d-98e3-77646fcb2ef8, upload has finished false
2024.11.04 23:20:41.314970 [ 3702 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/zwu/dxxyngidstbguliqasdynisqpyvgs, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2c6d4393-88fb-446d-98e3-77646fcb2ef8, upload has finished false
2024.11.04 23:20:41.319654 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 57 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:41.329961 [ 7099 ] {bc7179d6-31d0-48a0-a5f7-ba9d21f2c27a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function regexpQuoteMeta doesn't match: passed 3, should be 1: In scope SELECT regexpQuoteMeta(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT regexpQuoteMeta(NULL, NULL, NULL);)
2024.11.04 23:20:41.332149 [ 7099 ] {bc7179d6-31d0-48a0-a5f7-ba9d21f2c27a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function regexpQuoteMeta doesn't match: passed 3, should be 1: In scope SELECT regexpQuoteMeta(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:41.348667 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_116_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:42.106893 [ 7099 ] {f9258856-4113-4f42-b938-7ee8860451e2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpret doesn't match: passed 3, should be 2: In scope SELECT reinterpret(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpret(NULL, NULL, NULL);)
2024.11.04 23:20:42.108872 [ 7099 ] {f9258856-4113-4f42-b938-7ee8860451e2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpret doesn't match: passed 3, should be 2: In scope SELECT reinterpret(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.132232 [ 7099 ] {f9916866-63c1-42a5-a9d2-8c263aade798} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDate doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsDate(NULL, NULL, NULL);)
2024.11.04 23:20:42.134462 [ 7099 ] {f9916866-63c1-42a5-a9d2-8c263aade798} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDate doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.158094 [ 7099 ] {8ab74759-3b31-4aab-96d5-84eb2eb8ff01} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDateTime doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDateTime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsDateTime(NULL, NULL, NULL);)
2024.11.04 23:20:42.160448 [ 7099 ] {8ab74759-3b31-4aab-96d5-84eb2eb8ff01} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDateTime doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDateTime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.183436 [ 7099 ] {7f152ebf-7d70-47ed-911c-b4d9fa3ecf35} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFixedString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFixedString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsFixedString(NULL, NULL, NULL);)
2024.11.04 23:20:42.185784 [ 7099 ] {7f152ebf-7d70-47ed-911c-b4d9fa3ecf35} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFixedString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFixedString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.208116 [ 7099 ] {d8cfe9d2-7dc1-462c-bee3-0b468382554b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsFloat32(NULL, NULL, NULL);)
2024.11.04 23:20:42.290723 [ 7099 ] {d8cfe9d2-7dc1-462c-bee3-0b468382554b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.384990 [ 7099 ] {208346e6-e029-4270-ae75-0652a094419d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsFloat64(NULL, NULL, NULL);)
2024.11.04 23:20:42.387123 [ 7099 ] {208346e6-e029-4270-ae75-0652a094419d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.407347 [ 3234 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.407405 [ 3582 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.407490 [ 3247 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.407563 [ 3242 ] {} <Warning> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.407670 [ 3269 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.407777 [ 3272 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.407970 [ 3281 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.408075 [ 3246 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:42.408499 [ 3794 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/task_queue/ddl/. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:384: zkutil::ZooKeeper::createAncestors(String const&) @ 0x000000003c03f674
5. ./build_docker/./src/Interpreters/DDLWorker.cpp:0: DB::DDLWorker::initializeMainThread() @ 0x0000000032b57b01
6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1141: DB::DDLWorker::runMainThread() @ 0x0000000032b0bb4d
7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x0000000032b5e0c7
8. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000032b5df1f
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:42.409086 [ 7099 ] {afacf10a-be09-488b-9dd0-cb1095f96d8e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt128(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsInt128(NULL, NULL, NULL);)
2024.11.04 23:20:42.409351 [ 3614 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Operation timeout, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:42.411116 [ 7099 ] {afacf10a-be09-488b-9dd0-cb1095f96d8e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt128(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.411137 [ 3614 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1329: Coordination::ZooKeeper::exists(String const&, std::function<void (Coordination::ExistsResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c1618bf
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c045681
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c04483a
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:514: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045b90
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
10. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
11. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
13. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
15. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
16. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:42.433096 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:42.437140 [ 7099 ] {5e3ce88a-8d2f-4eff-a73d-8b610cf316a6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsInt16(NULL, NULL, NULL);)
2024.11.04 23:20:42.438994 [ 7099 ] {5e3ce88a-8d2f-4eff-a73d-8b610cf316a6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.462441 [ 7099 ] {c3e3e08a-8229-4ced-8dbe-86fbab7d7266} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsInt256(NULL, NULL, NULL);)
2024.11.04 23:20:42.502182 [ 7099 ] {c3e3e08a-8229-4ced-8dbe-86fbab7d7266} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.536583 [ 7099 ] {284f59f6-fff9-4c7b-baa8-bc2501d83ee3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsInt32(NULL, NULL, NULL);)
2024.11.04 23:20:42.539423 [ 7099 ] {284f59f6-fff9-4c7b-baa8-bc2501d83ee3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.576742 [ 7099 ] {5736e36d-d4bd-467f-94bf-e448960f5456} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsInt64(NULL, NULL, NULL);)
2024.11.04 23:20:42.579438 [ 7099 ] {5736e36d-d4bd-467f-94bf-e448960f5456} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.606197 [ 7099 ] {e02b7647-8aa9-4579-b8fb-eb86a4761c82} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsInt8(NULL, NULL, NULL);)
2024.11.04 23:20:42.608948 [ 7099 ] {e02b7647-8aa9-4579-b8fb-eb86a4761c82} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.635692 [ 7099 ] {eb61aa48-5f37-41b0-b595-851dd4bc4f15} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsString(NULL, NULL, NULL);)
2024.11.04 23:20:42.639736 [ 7099 ] {eb61aa48-5f37-41b0-b595-851dd4bc4f15} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.667115 [ 7099 ] {5ca19287-e124-4b2d-b138-2ab7cf2fc82c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt128(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsUInt128(NULL, NULL, NULL);)
2024.11.04 23:20:42.669248 [ 7099 ] {5ca19287-e124-4b2d-b138-2ab7cf2fc82c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt128(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.692078 [ 7099 ] {67fc0e0c-93b1-4639-9416-ec6bfd652f71} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsUInt16(NULL, NULL, NULL);)
2024.11.04 23:20:42.694675 [ 7099 ] {67fc0e0c-93b1-4639-9416-ec6bfd652f71} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt16(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.750912 [ 7099 ] {52ecd484-36ec-4bc6-be4e-37f3c1b9b5de} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsUInt256(NULL, NULL, NULL);)
2024.11.04 23:20:42.753637 [ 7099 ] {52ecd484-36ec-4bc6-be4e-37f3c1b9b5de} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt256(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.777728 [ 7099 ] {05e96917-c758-472b-9be3-0d28aa5bb355} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsUInt32(NULL, NULL, NULL);)
2024.11.04 23:20:42.780033 [ 7099 ] {05e96917-c758-472b-9be3-0d28aa5bb355} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt32(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.862724 [ 7099 ] {6db4c3af-e9cb-4723-8509-08d5884c95b8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsUInt64(NULL, NULL, NULL);)
2024.11.04 23:20:42.865235 [ 7099 ] {6db4c3af-e9cb-4723-8509-08d5884c95b8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt64(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:42.965228 [ 7099 ] {83bcd5cb-c54a-4d14-9d13-ba4f7f48a9dd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsUInt8(NULL, NULL, NULL);)
2024.11.04 23:20:42.967573 [ 7099 ] {83bcd5cb-c54a-4d14-9d13-ba4f7f48a9dd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:43.020002 [ 7099 ] {c53cdc30-71f3-428e-97c3-2ae9404dc06e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUUID doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUUID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reinterpretAsUUID(NULL, NULL, NULL);)
2024.11.04 23:20:43.022636 [ 7099 ] {c53cdc30-71f3-428e-97c3-2ae9404dc06e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUUID doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUUID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:43.049867 [ 7099 ] {fb76484d-93eb-4a47-9093-7391b1afb049} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function repeat doesn't match: passed 3, should be 2: In scope SELECT repeat(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT repeat(NULL, NULL, NULL);)
2024.11.04 23:20:43.051756 [ 7099 ] {fb76484d-93eb-4a47-9093-7391b1afb049} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function repeat doesn't match: passed 3, should be 2: In scope SELECT repeat(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:43.610992 [ 7099 ] {ac9f653b-9b2e-418c-967a-789754cc1286} <Error> executeQuery: Code: 43. DB::Exception: Argument 2 for function replicate must be array.: In scope SELECT replicate(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT replicate(NULL, NULL, NULL);)
2024.11.04 23:20:43.613449 [ 7099 ] {ac9f653b-9b2e-418c-967a-789754cc1286} <Error> TCPHandler: Code: 43. DB::Exception: Argument 2 for function replicate must be array.: In scope SELECT replicate(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:43.732982 [ 7099 ] {ecbd6fae-4b6a-4b60-9eb6-250c0a7d5800} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reverse doesn't match: passed 3, should be 1: In scope SELECT reverse(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reverse(NULL, NULL, NULL);)
2024.11.04 23:20:43.734863 [ 7099 ] {ecbd6fae-4b6a-4b60-9eb6-250c0a7d5800} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reverse doesn't match: passed 3, should be 1: In scope SELECT reverse(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:43.841545 [ 7099 ] {bf435906-2ceb-40d2-80c0-50a8586ddc6a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function reverseUTF8 doesn't match: passed 3, should be 1: In scope SELECT reverseUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT reverseUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:43.843815 [ 7099 ] {bf435906-2ceb-40d2-80c0-50a8586ddc6a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function reverseUTF8 doesn't match: passed 3, should be 1: In scope SELECT reverseUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:43.848461 [ 3949 ] {59ae75bd-dbcf-4325-a52f-a52e1992ea09} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45502) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:43.851719 [ 3949 ] {59ae75bd-dbcf-4325-a52f-a52e1992ea09} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:43.865792 [ 7099 ] {b955a7b0-9421-4bea-8cd0-3caf6c53ce99} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function revision doesn't match: passed 3, should be 0: In scope SELECT revision(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT revision(NULL, NULL, NULL);)
2024.11.04 23:20:43.868024 [ 7099 ] {b955a7b0-9421-4bea-8cd0-3caf6c53ce99} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function revision doesn't match: passed 3, should be 0: In scope SELECT revision(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:43.889923 [ 7099 ] {75358406-5bfb-4400-9b74-c8c32ad0f5dd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function right doesn't match: passed 3, should be 2: In scope SELECT right(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT right(NULL, NULL, NULL);)
2024.11.04 23:20:43.891882 [ 7099 ] {75358406-5bfb-4400-9b74-c8c32ad0f5dd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function right doesn't match: passed 3, should be 2: In scope SELECT right(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.068544 [ 7099 ] {b5a99843-45f4-44f8-97be-3886ddcc01c9} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function rightUTF8 doesn't match: passed 3, should be 2: In scope SELECT rightUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT rightUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:44.070555 [ 7099 ] {b5a99843-45f4-44f8-97be-3886ddcc01c9} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function rightUTF8 doesn't match: passed 3, should be 2: In scope SELECT rightUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.192314 [ 7099 ] {aaeefe4b-d6cb-4e3c-8e3f-6506a2fd8fb7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function roundAge doesn't match: passed 3, should be 1: In scope SELECT roundAge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT roundAge(NULL, NULL, NULL);)
2024.11.04 23:20:44.194253 [ 7099 ] {aaeefe4b-d6cb-4e3c-8e3f-6506a2fd8fb7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundAge doesn't match: passed 3, should be 1: In scope SELECT roundAge(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.255860 [ 7099 ] {d031c94f-4513-4370-adbe-b868a6fc8b93} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function roundDown doesn't match: passed 3, should be 2: In scope SELECT roundDown(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT roundDown(NULL, NULL, NULL);)
2024.11.04 23:20:44.258119 [ 7099 ] {d031c94f-4513-4370-adbe-b868a6fc8b93} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundDown doesn't match: passed 3, should be 2: In scope SELECT roundDown(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.279393 [ 7099 ] {f5d17a1e-c84b-4383-a677-4a383b521f82} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function roundDuration doesn't match: passed 3, should be 1: In scope SELECT roundDuration(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT roundDuration(NULL, NULL, NULL);)
2024.11.04 23:20:44.281760 [ 7099 ] {f5d17a1e-c84b-4383-a677-4a383b521f82} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundDuration doesn't match: passed 3, should be 1: In scope SELECT roundDuration(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.403950 [ 7099 ] {6f7434b1-24e4-41b8-8bd1-f309cffa0abb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function roundToExp2 doesn't match: passed 3, should be 1: In scope SELECT roundToExp2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT roundToExp2(NULL, NULL, NULL);)
2024.11.04 23:20:44.406160 [ 7099 ] {6f7434b1-24e4-41b8-8bd1-f309cffa0abb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundToExp2 doesn't match: passed 3, should be 1: In scope SELECT roundToExp2(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.438129 [ 7099 ] {4a5f1f9c-cc23-4896-b481-6f032e2751a7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function rowNumberInAllBlocks doesn't match: passed 3, should be 0: In scope SELECT rowNumberInAllBlocks(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT rowNumberInAllBlocks(NULL, NULL, NULL);)
2024.11.04 23:20:44.440801 [ 7099 ] {4a5f1f9c-cc23-4896-b481-6f032e2751a7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function rowNumberInAllBlocks doesn't match: passed 3, should be 0: In scope SELECT rowNumberInAllBlocks(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.463475 [ 7099 ] {7bcdb028-3be1-45c4-affc-cf70d316f824} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function rowNumberInBlock doesn't match: passed 3, should be 0: In scope SELECT rowNumberInBlock(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT rowNumberInBlock(NULL, NULL, NULL);)
2024.11.04 23:20:44.465845 [ 7099 ] {7bcdb028-3be1-45c4-affc-cf70d316f824} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function rowNumberInBlock doesn't match: passed 3, should be 0: In scope SELECT rowNumberInBlock(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.672473 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=28). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:44.813749 [ 7099 ] {e8d26901-eea4-496c-b325-36bfa964949d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT row_number(NULL, NULL, NULL);)
2024.11.04 23:20:44.818689 [ 7099 ] {e8d26901-eea4-496c-b325-36bfa964949d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:44.927880 [ 7099 ] {af31c2b2-84f7-4693-91c9-1fc46fe32809} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function runningConcurrency doesn't match: passed 3, should be 2: In scope SELECT runningConcurrency(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT runningConcurrency(NULL, NULL, NULL);)
2024.11.04 23:20:44.930110 [ 7099 ] {af31c2b2-84f7-4693-91c9-1fc46fe32809} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function runningConcurrency doesn't match: passed 3, should be 2: In scope SELECT runningConcurrency(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:44.954680 [ 7099 ] {6a0977fd-1b70-4b7f-a93b-c4ff8b596a27} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function runningDifference doesn't match: passed 3, should be 1: In scope SELECT runningDifference(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT runningDifference(NULL, NULL, NULL);)
2024.11.04 23:20:44.956702 [ 7099 ] {6a0977fd-1b70-4b7f-a93b-c4ff8b596a27} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function runningDifference doesn't match: passed 3, should be 1: In scope SELECT runningDifference(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.061850 [ 7099 ] {5405a378-9742-47c5-95af-a30a984611ed} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function runningDifferenceStartingWithFirstValue doesn't match: passed 3, should be 1: In scope SELECT runningDifferenceStartingWithFirstValue(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT runningDifferenceStartingWithFirstValue(NULL, NULL, NULL);)
2024.11.04 23:20:45.063833 [ 7099 ] {5405a378-9742-47c5-95af-a30a984611ed} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function runningDifferenceStartingWithFirstValue doesn't match: passed 3, should be 1: In scope SELECT runningDifferenceStartingWithFirstValue(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.117494 [ 7099 ] {7d5d03e3-0f3f-410a-a747-f2110bc208a0} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function s2CapUnion doesn't match: passed 3, should be 4: In scope SELECT s2CapUnion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT s2CapUnion(NULL, NULL, NULL);)
2024.11.04 23:20:45.119399 [ 7099 ] {7d5d03e3-0f3f-410a-a747-f2110bc208a0} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2CapUnion doesn't match: passed 3, should be 4: In scope SELECT s2CapUnion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.204645 [ 7099 ] {92a20c2f-8f2c-44fe-be50-9a5fc3234afd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function s2CellsIntersect doesn't match: passed 3, should be 2: In scope SELECT s2CellsIntersect(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT s2CellsIntersect(NULL, NULL, NULL);)
2024.11.04 23:20:45.206577 [ 7099 ] {92a20c2f-8f2c-44fe-be50-9a5fc3234afd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2CellsIntersect doesn't match: passed 3, should be 2: In scope SELECT s2CellsIntersect(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.229119 [ 7099 ] {16bdd30e-5d6b-4c44-814e-c3587d53a1b4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function s2GetNeighbors doesn't match: passed 3, should be 1: In scope SELECT s2GetNeighbors(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT s2GetNeighbors(NULL, NULL, NULL);)
2024.11.04 23:20:45.230999 [ 7099 ] {16bdd30e-5d6b-4c44-814e-c3587d53a1b4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2GetNeighbors doesn't match: passed 3, should be 1: In scope SELECT s2GetNeighbors(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.337790 [ 7099 ] {9fa40fc8-7d88-4281-805a-e96c2fb6c730} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function s2RectIntersection doesn't match: passed 3, should be 4: In scope SELECT s2RectIntersection(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT s2RectIntersection(NULL, NULL, NULL);)
2024.11.04 23:20:45.340020 [ 7099 ] {9fa40fc8-7d88-4281-805a-e96c2fb6c730} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2RectIntersection doesn't match: passed 3, should be 4: In scope SELECT s2RectIntersection(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.362678 [ 7099 ] {b167d949-da0e-418a-8cbd-3b7b6b759f08} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function s2RectUnion doesn't match: passed 3, should be 4: In scope SELECT s2RectUnion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT s2RectUnion(NULL, NULL, NULL);)
2024.11.04 23:20:45.364693 [ 7099 ] {b167d949-da0e-418a-8cbd-3b7b6b759f08} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2RectUnion doesn't match: passed 3, should be 4: In scope SELECT s2RectUnion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.389893 [ 7099 ] {1881387d-fd19-4d91-a687-6f6d9b564ab1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function s2ToGeo doesn't match: passed 3, should be 1: In scope SELECT s2ToGeo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT s2ToGeo(NULL, NULL, NULL);)
2024.11.04 23:20:45.391815 [ 7099 ] {1881387d-fd19-4d91-a687-6f6d9b564ab1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2ToGeo doesn't match: passed 3, should be 1: In scope SELECT s2ToGeo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.416155 [ 7099 ] {764c660b-b567-4bcf-8f7d-27e26e9ddb82} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT scalarProduct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT scalarProduct(NULL, NULL, NULL);)
2024.11.04 23:20:45.418110 [ 7099 ] {764c660b-b567-4bcf-8f7d-27e26e9ddb82} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT scalarProduct(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.462014 [ 7099 ] {d4e40c7c-2724-4f5a-bd78-d59f15417c76} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sequenceCount(NULL, NULL, NULL);)
2024.11.04 23:20:45.466288 [ 7099 ] {d4e40c7c-2724-4f5a-bd78-d59f15417c76} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:45.636811 [ 7099 ] {aa6ba432-ad71-4bba-87f7-c1512335207a} <Error> executeQuery: Code: 63. DB::Exception: Aggregate function sequenceNextNode is experimental. Set `allow_experimental_funnel_functions` setting to enable it. (UNKNOWN_AGGREGATE_FUNCTION) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sequenceNextNode(NULL, NULL, NULL);)
2024.11.04 23:20:45.639075 [ 7099 ] {aa6ba432-ad71-4bba-87f7-c1512335207a} <Error> TCPHandler: Code: 63. DB::Exception: Aggregate function sequenceNextNode is experimental. Set `allow_experimental_funnel_functions` setting to enable it. (UNKNOWN_AGGREGATE_FUNCTION)
2024.11.04 23:20:45.732009 [ 7099 ] {26ad72d6-1eef-490c-9c9f-b1b886b5d915} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function seriesDecomposeSTL doesn't match: passed 3, should be 2: In scope SELECT seriesDecomposeSTL(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT seriesDecomposeSTL(NULL, NULL, NULL);)
2024.11.04 23:20:45.734592 [ 7099 ] {26ad72d6-1eef-490c-9c9f-b1b886b5d915} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function seriesDecomposeSTL doesn't match: passed 3, should be 2: In scope SELECT seriesDecomposeSTL(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.794007 [ 7099 ] {e389638f-8f60-4366-afba-1fbce5e7a41b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function seriesPeriodDetectFFT doesn't match: passed 3, should be 1: In scope SELECT seriesPeriodDetectFFT(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT seriesPeriodDetectFFT(NULL, NULL, NULL);)
2024.11.04 23:20:45.796027 [ 7099 ] {e389638f-8f60-4366-afba-1fbce5e7a41b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function seriesPeriodDetectFFT doesn't match: passed 3, should be 1: In scope SELECT seriesPeriodDetectFFT(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.818754 [ 7099 ] {6ef01ef8-cb05-4e19-8baa-a627ec203e7e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimeZone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT serverTimeZone(NULL, NULL, NULL);)
2024.11.04 23:20:45.820716 [ 7099 ] {6ef01ef8-cb05-4e19-8baa-a627ec203e7e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimeZone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.843202 [ 7099 ] {739a6aa8-e107-4501-bdc6-0777bc30e66c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimezone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT serverTimezone(NULL, NULL, NULL);)
2024.11.04 23:20:45.845313 [ 7099 ] {739a6aa8-e107-4501-bdc6-0777bc30e66c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimezone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.866589 [ 7099 ] {bede2553-4bb3-47c7-a859-98fb03841b8e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function serverUUID doesn't match: passed 3, should be 0: In scope SELECT serverUUID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT serverUUID(NULL, NULL, NULL);)
2024.11.04 23:20:45.868317 [ 7099 ] {bede2553-4bb3-47c7-a859-98fb03841b8e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function serverUUID doesn't match: passed 3, should be 0: In scope SELECT serverUUID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.887582 [ 7099 ] {a4fcd353-f5b9-4743-a6f9-6e21ef6d2b03} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function shardCount doesn't match: passed 3, should be 0: In scope SELECT shardCount(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT shardCount(NULL, NULL, NULL);)
2024.11.04 23:20:45.889568 [ 7099 ] {a4fcd353-f5b9-4743-a6f9-6e21ef6d2b03} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function shardCount doesn't match: passed 3, should be 0: In scope SELECT shardCount(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:45.955682 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_116_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:45.986377 [ 7099 ] {40ce1c2a-57f9-4931-8a90-9983f22a3c61} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function shardNum doesn't match: passed 3, should be 0: In scope SELECT shardNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT shardNum(NULL, NULL, NULL);)
2024.11.04 23:20:45.988542 [ 7099 ] {40ce1c2a-57f9-4931-8a90-9983f22a3c61} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function shardNum doesn't match: passed 3, should be 0: In scope SELECT shardNum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.009780 [ 7099 ] {7b585b14-2028-4521-8a54-1a12aca01726} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function showCertificate doesn't match: passed 3, should be 0: In scope SELECT showCertificate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT showCertificate(NULL, NULL, NULL);)
2024.11.04 23:20:46.011755 [ 7099 ] {7b585b14-2028-4521-8a54-1a12aca01726} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function showCertificate doesn't match: passed 3, should be 0: In scope SELECT showCertificate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.091066 [ 7099 ] {189cecd3-8084-4ae6-bbf1-fbb7d5e7e924} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sigmoid doesn't match: passed 3, should be 1: In scope SELECT sigmoid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sigmoid(NULL, NULL, NULL);)
2024.11.04 23:20:46.093359 [ 7099 ] {189cecd3-8084-4ae6-bbf1-fbb7d5e7e924} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sigmoid doesn't match: passed 3, should be 1: In scope SELECT sigmoid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.116547 [ 7099 ] {6d7919d3-ea31-4937-a24c-fabe006ed137} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sign doesn't match: passed 3, should be 1: In scope SELECT sign(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sign(NULL, NULL, NULL);)
2024.11.04 23:20:46.118719 [ 7099 ] {6d7919d3-ea31-4937-a24c-fabe006ed137} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sign doesn't match: passed 3, should be 1: In scope SELECT sign(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.160579 [ 7099 ] {50dac452-ae05-49be-9895-1b53d48ec529} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractBool(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT simpleJSONExtractBool(NULL, NULL, NULL);)
2024.11.04 23:20:46.162750 [ 7099 ] {50dac452-ae05-49be-9895-1b53d48ec529} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractBool(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.187465 [ 7099 ] {ef34596d-d9b2-43fc-ad52-df2e91f7465a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractFloat(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT simpleJSONExtractFloat(NULL, NULL, NULL);)
2024.11.04 23:20:46.190426 [ 7099 ] {ef34596d-d9b2-43fc-ad52-df2e91f7465a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractFloat(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.223928 [ 7099 ] {78f51923-748f-4ffb-a78f-eb7fab784368} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT simpleJSONExtractInt(NULL, NULL, NULL);)
2024.11.04 23:20:46.226229 [ 7099 ] {78f51923-748f-4ffb-a78f-eb7fab784368} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.258640 [ 7099 ] {a635824a-6d79-46d0-b08c-9255d45b8446} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractRaw(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT simpleJSONExtractRaw(NULL, NULL, NULL);)
2024.11.04 23:20:46.260716 [ 7099 ] {a635824a-6d79-46d0-b08c-9255d45b8446} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractRaw(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.334511 [ 7099 ] {5fea6186-a603-46fc-bd56-569d22f02547} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT simpleJSONExtractString(NULL, NULL, NULL);)
2024.11.04 23:20:46.336667 [ 7099 ] {5fea6186-a603-46fc-bd56-569d22f02547} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.362161 [ 7099 ] {678d8c77-265b-4b75-af7f-9a0c1c9d14fc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractUInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT simpleJSONExtractUInt(NULL, NULL, NULL);)
2024.11.04 23:20:46.364412 [ 7099 ] {678d8c77-265b-4b75-af7f-9a0c1c9d14fc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractUInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.383741 [ 7099 ] {1bb668c5-f8aa-4407-9728-511f001ba49f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT simpleJSONHas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT simpleJSONHas(NULL, NULL, NULL);)
2024.11.04 23:20:46.385498 [ 7099 ] {1bb668c5-f8aa-4407-9728-511f001ba49f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT simpleJSONHas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.534366 [ 7099 ] {35bed567-1c7b-47ac-b616-b1dfe918b87f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sin doesn't match: passed 3, should be 1: In scope SELECT sin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sin(NULL, NULL, NULL);)
2024.11.04 23:20:46.536914 [ 7099 ] {35bed567-1c7b-47ac-b616-b1dfe918b87f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sin doesn't match: passed 3, should be 1: In scope SELECT sin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:46.587889 [ 7099 ] {a6758cca-7139-416b-85a1-a2bf9dfaad1a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT singleValueOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:46.592664 [ 7099 ] {a6758cca-7139-416b-85a1-a2bf9dfaad1a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:46.615616 [ 7099 ] {a4450bdf-cd63-480a-aa9d-c73148ac76ea} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sinh doesn't match: passed 3, should be 1: In scope SELECT sinh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sinh(NULL, NULL, NULL);)
2024.11.04 23:20:46.617768 [ 7099 ] {a4450bdf-cd63-480a-aa9d-c73148ac76ea} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sinh doesn't match: passed 3, should be 1: In scope SELECT sinh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:47.085580 [ 3949 ] {8d93a2a6-0b2d-44f3-95d7-038ec84ca4cd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45532) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:47.086701 [ 7099 ] {4e663902-2a3a-4cd8-89b4-388ad35da085} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sleep doesn't match: passed 3, should be 1: In scope SELECT sleep(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sleep(NULL, NULL, NULL);)
2024.11.04 23:20:47.088685 [ 3949 ] {8d93a2a6-0b2d-44f3-95d7-038ec84ca4cd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:47.088677 [ 7099 ] {4e663902-2a3a-4cd8-89b4-388ad35da085} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sleep doesn't match: passed 3, should be 1: In scope SELECT sleep(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:47.110398 [ 7099 ] {9ed5cbf5-874c-49f4-a40d-e462dd436098} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sleepEachRow doesn't match: passed 3, should be 1: In scope SELECT sleepEachRow(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sleepEachRow(NULL, NULL, NULL);)
2024.11.04 23:20:47.112649 [ 7099 ] {9ed5cbf5-874c-49f4-a40d-e462dd436098} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sleepEachRow doesn't match: passed 3, should be 1: In scope SELECT sleepEachRow(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:47.143232 [ 7099 ] {6a856d6f-8b11-47ca-b772-c513406010ed} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT snowflakeToDateTime(NULL, NULL, NULL);)
2024.11.04 23:20:47.145649 [ 7099 ] {6a856d6f-8b11-47ca-b772-c513406010ed} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:47.167045 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/mjy/vfdehkclugcdcgwuyvcrxhwluegej, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2350a26a-e466-4efb-ae28-00f481e8b8f0, upload has finished false.
2024.11.04 23:20:47.167273 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/mjy/vfdehkclugcdcgwuyvcrxhwluegej, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2350a26a-e466-4efb-ae28-00f481e8b8f0, upload has finished false
2024.11.04 23:20:47.187696 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/mjy/vfdehkclugcdcgwuyvcrxhwluegej, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2350a26a-e466-4efb-ae28-00f481e8b8f0, upload has finished false
2024.11.04 23:20:47.192358 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 10 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:47.264428 [ 7099 ] {1080277d-9202-4906-8451-098a5a0da51e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function soundex doesn't match: passed 3, should be 1: In scope SELECT soundex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT soundex(NULL, NULL, NULL);)
2024.11.04 23:20:47.266504 [ 7099 ] {1080277d-9202-4906-8451-098a5a0da51e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function soundex doesn't match: passed 3, should be 1: In scope SELECT soundex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:47.285677 [ 7099 ] {d760ed01-af35-44d8-8444-598b1fe673d7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function space doesn't match: passed 3, should be 1: In scope SELECT space(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT space(NULL, NULL, NULL);)
2024.11.04 23:20:47.287485 [ 7099 ] {d760ed01-af35-44d8-8444-598b1fe673d7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function space doesn't match: passed 3, should be 1: In scope SELECT space(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:47.337507 [ 7099 ] {7a5bda24-e7d3-46e0-9f9a-d72c88919521} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sparkBar(NULL, NULL, NULL);)
2024.11.04 23:20:47.341625 [ 7099 ] {7a5bda24-e7d3-46e0-9f9a-d72c88919521} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:47.450270 [ 7099 ] {abc6f21d-71fb-4387-b7aa-c84074d25e37} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=32, jobs=32). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sparkbar(NULL, NULL, NULL);)
2024.11.04 23:20:47.454412 [ 7099 ] {abc6f21d-71fb-4387-b7aa-c84074d25e37} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=32, jobs=32). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:47.694418 [ 7099 ] {dd9dc176-4850-493a-becf-b25100bb2ac8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT splitByRegexp(NULL, NULL, NULL);)
2024.11.04 23:20:47.696917 [ 7099 ] {dd9dc176-4850-493a-becf-b25100bb2ac8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:47.971450 [ 7099 ] {9aba9228-bb8b-436e-b6ff-e725b6cd231a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sqidDecode doesn't match: passed 3, should be 1: In scope SELECT sqidDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sqidDecode(NULL, NULL, NULL);)
2024.11.04 23:20:47.973606 [ 7099 ] {9aba9228-bb8b-436e-b6ff-e725b6cd231a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sqidDecode doesn't match: passed 3, should be 1: In scope SELECT sqidDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.026505 [ 7099 ] {74b5a90d-95c8-4a50-8e83-711d72652eb4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function sqrt doesn't match: passed 3, should be 1: In scope SELECT sqrt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sqrt(NULL, NULL, NULL);)
2024.11.04 23:20:48.028536 [ 7099 ] {74b5a90d-95c8-4a50-8e83-711d72652eb4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function sqrt doesn't match: passed 3, should be 1: In scope SELECT sqrt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.037972 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:48.047956 [ 7099 ] {2d7cdf4f-2b53-45ec-b088-304b0cbc701e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function startsWith doesn't match: passed 3, should be 2: In scope SELECT startsWith(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT startsWith(NULL, NULL, NULL);)
2024.11.04 23:20:48.049920 [ 7099 ] {2d7cdf4f-2b53-45ec-b088-304b0cbc701e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function startsWith doesn't match: passed 3, should be 2: In scope SELECT startsWith(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.071468 [ 7099 ] {032d718e-24f3-48cb-857f-3038dd398d0c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function startsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT startsWithUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT startsWithUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:48.073421 [ 7099 ] {032d718e-24f3-48cb-857f-3038dd398d0c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function startsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT startsWithUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.186880 [ 7099 ] {d7e51e92-1722-4025-a6f8-682f6cc9c89a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=35). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT stddevPop(NULL, NULL, NULL);)
2024.11.04 23:20:48.192558 [ 7099 ] {d7e51e92-1722-4025-a6f8-682f6cc9c89a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=35, jobs=35). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:48.496623 [ 7099 ] {18b41b70-6d3d-43e3-bb88-86591e407e3a} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'stem' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT stem(NULL, NULL, NULL);)
2024.11.04 23:20:48.498479 [ 7099 ] {18b41b70-6d3d-43e3-bb88-86591e407e3a} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'stem' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:48.736998 [ 7099 ] {e0135077-74c2-4b33-bc2f-df2ee2458b99} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT stringJaccardIndex(NULL, NULL, NULL);)
2024.11.04 23:20:48.739249 [ 7099 ] {e0135077-74c2-4b33-bc2f-df2ee2458b99} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.837067 [ 7099 ] {c84550ad-4ac3-432b-b043-baa26e0d4d36} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndexUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT stringJaccardIndexUTF8(NULL, NULL, NULL);)
2024.11.04 23:20:48.839013 [ 7099 ] {c84550ad-4ac3-432b-b043-baa26e0d4d36} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndexUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.860111 [ 7099 ] {f7a2167b-5496-40d7-be3d-a5a6006a0ba7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function stringToH3 doesn't match: passed 3, should be 1: In scope SELECT stringToH3(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT stringToH3(NULL, NULL, NULL);)
2024.11.04 23:20:48.862188 [ 7099 ] {f7a2167b-5496-40d7-be3d-a5a6006a0ba7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function stringToH3 doesn't match: passed 3, should be 1: In scope SELECT stringToH3(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.883309 [ 7099 ] {ca66b23c-5026-45bd-90da-d0ed8b4ada72} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function structureToCapnProtoSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToCapnProtoSchema(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT structureToCapnProtoSchema(NULL, NULL, NULL);)
2024.11.04 23:20:48.885302 [ 7099 ] {ca66b23c-5026-45bd-90da-d0ed8b4ada72} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function structureToCapnProtoSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToCapnProtoSchema(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:48.908361 [ 7099 ] {8df677e0-42df-453f-8aa3-2724b277e811} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function structureToProtobufSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToProtobufSchema(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT structureToProtobufSchema(NULL, NULL, NULL);)
2024.11.04 23:20:48.910358 [ 7099 ] {8df677e0-42df-453f-8aa3-2724b277e811} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function structureToProtobufSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToProtobufSchema(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:49.189109 [ 3692 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded hard memory limit (14.54 GiB)
2024.11.04 23:20:49.263591 [ 7099 ] {3dad1c55-9e37-42b0-9e28-752b6c93ddc7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function subDate doesn't match: passed 3, should be 2: In scope SELECT subDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT subDate(NULL, NULL, NULL);)
2024.11.04 23:20:49.266359 [ 7099 ] {3dad1c55-9e37-42b0-9e28-752b6c93ddc7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function subDate doesn't match: passed 3, should be 2: In scope SELECT subDate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:49.284791 [ 3949 ] {1c5862c0-3cb3-4a23-ae61-9027c284acff} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45556) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:49.288202 [ 3949 ] {1c5862c0-3cb3-4a23-ae61-9027c284acff} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:49.477062 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 49 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:49.585847 [ 7099 ] {3c52f853-5a42-461d-b081-bb166e94e837} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function subtractInterval doesn't match: passed 3, should be 2: In scope SELECT subtractInterval(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT subtractInterval(NULL, NULL, NULL);)
2024.11.04 23:20:49.587662 [ 7099 ] {3c52f853-5a42-461d-b081-bb166e94e837} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function subtractInterval doesn't match: passed 3, should be 2: In scope SELECT subtractInterval(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:49.896961 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:50.399304 [ 3949 ] {195e9919-3482-447e-b624-a0dce1d89e8b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:45558) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:50.404754 [ 3949 ] {195e9919-3482-447e-b624-a0dce1d89e8b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:50.517970 [ 7099 ] {d2540c85-b1fb-48dd-a918-fc744416db79} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function subtractTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT subtractTupleOfIntervals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT subtractTupleOfIntervals(NULL, NULL, NULL);)
2024.11.04 23:20:50.521333 [ 7099 ] {d2540c85-b1fb-48dd-a918-fc744416db79} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function subtractTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT subtractTupleOfIntervals(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:50.963121 [ 7099 ] {e5437965-68d7-4bc7-8cef-94f7e9e68a8b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sumKahan(NULL, NULL, NULL);)
2024.11.04 23:20:50.968469 [ 7099 ] {e5437965-68d7-4bc7-8cef-94f7e9e68a8b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:51.017561 [ 7099 ] {832be017-cf84-41bf-8cca-c89bc1e4778f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sumMapFiltered(NULL, NULL, NULL);)
2024.11.04 23:20:51.022114 [ 7099 ] {832be017-cf84-41bf-8cca-c89bc1e4778f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:51.163557 [ 7099 ] {9ab1ba03-71ef-4710-9ddf-15a30a744500} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sumMapFilteredWithOverflow(NULL, NULL, NULL);)
2024.11.04 23:20:51.168845 [ 7099 ] {9ab1ba03-71ef-4710-9ddf-15a30a744500} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:51.311811 [ 7099 ] {8e82f1ed-3e8d-48d9-8adc-a9fb2713089e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT sumMappedArrays(NULL, NULL, NULL);)
2024.11.04 23:20:51.316756 [ 7099 ] {8e82f1ed-3e8d-48d9-8adc-a9fb2713089e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:20:51.494978 [ 7099 ] {804cb2d0-8d47-42e1-b9b3-0da25ccedbbf} <Error> executeQuery: Code: 344. DB::Exception: Natural language processing function 'synonyms' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT synonyms(NULL, NULL, NULL);)
2024.11.04 23:20:51.496933 [ 7099 ] {804cb2d0-8d47-42e1-b9b3-0da25ccedbbf} <Error> TCPHandler: Code: 344. DB::Exception: Natural language processing function 'synonyms' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED)
2024.11.04 23:20:51.514840 [ 7099 ] {a722b73f-0721-42d8-b9ac-f4d91c597f21} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tan doesn't match: passed 3, should be 1: In scope SELECT tan(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tan(NULL, NULL, NULL);)
2024.11.04 23:20:51.516986 [ 7099 ] {a722b73f-0721-42d8-b9ac-f4d91c597f21} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tan doesn't match: passed 3, should be 1: In scope SELECT tan(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:51.529104 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:51.535771 [ 7099 ] {a7123e4d-9513-49cf-a7aa-3fdee76b4ae4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tanh doesn't match: passed 3, should be 1: In scope SELECT tanh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tanh(NULL, NULL, NULL);)
2024.11.04 23:20:51.537684 [ 7099 ] {a7123e4d-9513-49cf-a7aa-3fdee76b4ae4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tanh doesn't match: passed 3, should be 1: In scope SELECT tanh(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:51.555654 [ 7099 ] {dc7c2eca-fddd-4d65-b2d2-dad0a092240d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tcpPort doesn't match: passed 3, should be 0: In scope SELECT tcpPort(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tcpPort(NULL, NULL, NULL);)
2024.11.04 23:20:51.557518 [ 7099 ] {dc7c2eca-fddd-4d65-b2d2-dad0a092240d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tcpPort doesn't match: passed 3, should be 0: In scope SELECT tcpPort(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:51.574676 [ 7099 ] {de0b3dae-2551-4f10-b532-4921ab0d3769} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function test_function doesn't match: passed 3, should be 2: In scope SELECT test_function(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT test_function(NULL, NULL, NULL);)
2024.11.04 23:20:51.632062 [ 7099 ] {de0b3dae-2551-4f10-b532-4921ab0d3769} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function test_function doesn't match: passed 3, should be 2: In scope SELECT test_function(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:51.651805 [ 7099 ] {a16b0877-be5d-4b9a-8139-4cc0f1b38da2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tgamma doesn't match: passed 3, should be 1: In scope SELECT tgamma(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tgamma(NULL, NULL, NULL);)
2024.11.04 23:20:51.654108 [ 7099 ] {a16b0877-be5d-4b9a-8139-4cc0f1b38da2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tgamma doesn't match: passed 3, should be 1: In scope SELECT tgamma(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:51.902655 [ 7099 ] {5c9ad6b2-3fe5-462f-865f-7f0b1e04246a} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tid doesn't match: passed 3, should be 0: In scope SELECT tid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tid(NULL, NULL, NULL);)
2024.11.04 23:20:51.906361 [ 7099 ] {5c9ad6b2-3fe5-462f-865f-7f0b1e04246a} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tid doesn't match: passed 3, should be 0: In scope SELECT tid(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:51.940814 [ 7099 ] {7711784e-26f9-4ec0-9f02-92847e905850} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function timeDiff doesn't match: passed 3, should be 2: In scope SELECT timeDiff(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT timeDiff(NULL, NULL, NULL);)
2024.11.04 23:20:51.944149 [ 7099 ] {7711784e-26f9-4ec0-9f02-92847e905850} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function timeDiff doesn't match: passed 3, should be 2: In scope SELECT timeDiff(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.164421 [ 7099 ] {7b9567a5-ad88-4b32-8336-930d6843231d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timeZone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT timeZone(NULL, NULL, NULL);)
2024.11.04 23:20:52.167487 [ 7099 ] {7b9567a5-ad88-4b32-8336-930d6843231d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timeZone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.198854 [ 7099 ] {bd1d5358-92bf-4617-a3d1-a1576d518646} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timeZoneOf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT timeZoneOf(NULL, NULL, NULL);)
2024.11.04 23:20:52.201912 [ 7099 ] {bd1d5358-92bf-4617-a3d1-a1576d518646} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timeZoneOf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.445330 [ 7099 ] {bd7656f3-4c7d-43b3-9d84-8997ff94c854} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timezone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT timezone(NULL, NULL, NULL);)
2024.11.04 23:20:52.447274 [ 7099 ] {bd7656f3-4c7d-43b3-9d84-8997ff94c854} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timezone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.466923 [ 7099 ] {63f7925a-9cc8-422c-9681-b4997eec81a2} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timezoneOf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT timezoneOf(NULL, NULL, NULL);)
2024.11.04 23:20:52.468860 [ 7099 ] {63f7925a-9cc8-422c-9681-b4997eec81a2} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timezoneOf(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.523448 [ 7099 ] {dfd30da1-8053-4baa-afee-a1f33fff360c} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toBool doesn't match: passed 3, should be 1: In scope SELECT toBool(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toBool(NULL, NULL, NULL);)
2024.11.04 23:20:52.525890 [ 7099 ] {dfd30da1-8053-4baa-afee-a1f33fff360c} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toBool doesn't match: passed 3, should be 1: In scope SELECT toBool(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.546267 [ 7099 ] {b52ec22f-dd55-49f4-9d3b-6874f12c2202} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toColumnTypeName doesn't match: passed 3, should be 1: In scope SELECT toColumnTypeName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toColumnTypeName(NULL, NULL, NULL);)
2024.11.04 23:20:52.548091 [ 7099 ] {b52ec22f-dd55-49f4-9d3b-6874f12c2202} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toColumnTypeName doesn't match: passed 3, should be 1: In scope SELECT toColumnTypeName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.672764 [ 7099 ] {9dd1b13c-2c35-4cff-8d60-dcbef4d3af83} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toDate32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toDate32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDate32OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:52.674731 [ 7099 ] {9dd1b13c-2c35-4cff-8d60-dcbef4d3af83} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toDate32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toDate32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:52.774345 [ 7099 ] {fe99102b-fcb9-465f-85f3-ecbc2b4fda24} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toDateOrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toDateOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDateOrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:52.776493 [ 7099 ] {fe99102b-fcb9-465f-85f3-ecbc2b4fda24} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toDateOrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toDateOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:53.072701 [ 7099 ] {bc6071cd-84c4-48b7-90a5-42038424268e} <Error> executeQuery: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDateTime64OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDateTime64OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDateTime64OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:53.074661 [ 7099 ] {bc6071cd-84c4-48b7-90a5-42038424268e} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDateTime64OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDateTime64OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:53.249726 [ 7099 ] {ff8dcf52-6fc2-43c3-8c47-22824c1158ab} <Error> executeQuery: Code: 43. DB::Exception: Illegal type of argument #2 'timezone' of function toDateTimeOrDefault, expected const String, got Nullable(Nothing): In scope SELECT toDateTimeOrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDateTimeOrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:53.251810 [ 7099 ] {ff8dcf52-6fc2-43c3-8c47-22824c1158ab} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type of argument #2 'timezone' of function toDateTimeOrDefault, expected const String, got Nullable(Nothing): In scope SELECT toDateTimeOrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:53.559602 [ 7099 ] {ab80e1f6-7f9e-4a68-9601-b219e6f97843} <Error> executeQuery: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal128OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal128OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDecimal128OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:53.561984 [ 7099 ] {ab80e1f6-7f9e-4a68-9601-b219e6f97843} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal128OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal128OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:53.939739 [ 7099 ] {4945808f-22db-49ff-be69-a590dc72f06f} <Error> executeQuery: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal256OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal256OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDecimal256OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:53.942289 [ 7099 ] {4945808f-22db-49ff-be69-a590dc72f06f} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal256OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal256OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:54.134306 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:54.136026 [ 7099 ] {a6ccc400-93a4-43ca-b300-8b564d8f7876} <Error> executeQuery: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal32OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal32OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDecimal32OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:54.138153 [ 7099 ] {a6ccc400-93a4-43ca-b300-8b564d8f7876} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal32OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal32OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:54.262083 [ 7099 ] {06689206-984e-4390-86b4-14ea41e001e9} <Error> executeQuery: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal64OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal64OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDecimal64OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:54.264524 [ 7099 ] {06689206-984e-4390-86b4-14ea41e001e9} <Error> TCPHandler: Code: 43. DB::Exception: Illegal type of argument #2 'scale' of function toDecimal64OrDefault, expected const Integer, got Nullable(Nothing): In scope SELECT toDecimal64OrDefault(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:20:54.349820 [ 7099 ] {4e3e2af6-222a-47b6-9f31-3dd5ecb99f3f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 3, should be 2: In scope SELECT toDecimalString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toDecimalString(NULL, NULL, NULL);)
2024.11.04 23:20:54.351884 [ 7099 ] {4e3e2af6-222a-47b6-9f31-3dd5ecb99f3f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 3, should be 2: In scope SELECT toDecimalString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:54.373896 [ 7099 ] {15d11d70-4110-456f-8911-096e5dbc8a96} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toFixedString doesn't match: passed 3, should be 2: In scope SELECT toFixedString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toFixedString(NULL, NULL, NULL);)
2024.11.04 23:20:54.376580 [ 7099 ] {15d11d70-4110-456f-8911-096e5dbc8a96} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toFixedString doesn't match: passed 3, should be 2: In scope SELECT toFixedString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:54.563708 [ 7099 ] {b0a816d0-726c-4da1-966d-2bf4e23eb898} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toFloat32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toFloat32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toFloat32OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:54.566318 [ 7099 ] {b0a816d0-726c-4da1-966d-2bf4e23eb898} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toFloat32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toFloat32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:54.976408 [ 7099 ] {e630d51c-eb7f-4c69-a40b-bf06528a78d3} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toFloat64OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toFloat64OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toFloat64OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:54.978426 [ 7099 ] {e630d51c-eb7f-4c69-a40b-bf06528a78d3} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toFloat64OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toFloat64OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:55.196428 [ 7099 ] {429c8259-70dd-4b46-b0ae-d999ee3746b1} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toIPv4OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toIPv4OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toIPv4OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:55.198697 [ 7099 ] {429c8259-70dd-4b46-b0ae-d999ee3746b1} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toIPv4OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toIPv4OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:55.362716 [ 7099 ] {e7893e3d-f5cb-47c3-9914-836da8d70f04} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toIPv6OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toIPv6OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toIPv6OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:55.364804 [ 7099 ] {e7893e3d-f5cb-47c3-9914-836da8d70f04} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toIPv6OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toIPv6OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:55.519061 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ldc/ubqmxagbwjktjibkdmhpfcmqqgjnw, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5b6e00e3-c59b-41fe-a8de-ead7a3e2c46d, upload has finished false.
2024.11.04 23:20:55.519234 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ldc/ubqmxagbwjktjibkdmhpfcmqqgjnw, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5b6e00e3-c59b-41fe-a8de-ead7a3e2c46d, upload has finished false
2024.11.04 23:20:55.574220 [ 7099 ] {59930f8c-7851-400b-b664-05fe15b0bb08} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toInt128OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt128OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toInt128OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:55.607828 [ 7099 ] {59930f8c-7851-400b-b664-05fe15b0bb08} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toInt128OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt128OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:55.660615 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ldc/ubqmxagbwjktjibkdmhpfcmqqgjnw, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5b6e00e3-c59b-41fe-a8de-ead7a3e2c46d, upload has finished false
2024.11.04 23:20:55.665790 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 2 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:56.021145 [ 7099 ] {004d8da0-000b-474f-a8fa-40ecf0dedc27} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toInt16OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt16OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toInt16OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:56.023475 [ 7099 ] {004d8da0-000b-474f-a8fa-40ecf0dedc27} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toInt16OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt16OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:56.226416 [ 7099 ] {e3126e03-2959-4b07-b419-ccb2e24a0306} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toInt256OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt256OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toInt256OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:56.228526 [ 7099 ] {e3126e03-2959-4b07-b419-ccb2e24a0306} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toInt256OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt256OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:56.459719 [ 7099 ] {ff17577f-2afc-4a79-9074-deb78d386247} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toInt32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toInt32OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:56.462110 [ 7099 ] {ff17577f-2afc-4a79-9074-deb78d386247} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toInt32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:56.609588 [ 7099 ] {de81842a-3a99-4bab-a2b2-9990c3254b5f} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toInt64OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt64OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toInt64OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:56.611990 [ 7099 ] {de81842a-3a99-4bab-a2b2-9990c3254b5f} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toInt64OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt64OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:56.810039 [ 7099 ] {cef923ec-6043-4e4e-b706-6fa4a7fbd19b} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toInt8OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt8OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toInt8OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:56.812357 [ 7099 ] {cef923ec-6043-4e4e-b706-6fa4a7fbd19b} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toInt8OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toInt8OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:57.047988 [ 3949 ] {208493c6-b0ec-4f78-b615-482f3e6329d1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:47324) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:57.053487 [ 3949 ] {208493c6-b0ec-4f78-b615-482f3e6329d1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:20:57.352765 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:57.627343 [ 7099 ] {1f6f3874-ca4c-4ff0-b64d-f72ebbdaf681} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toJSONString doesn't match: passed 3, should be 1: In scope SELECT toJSONString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toJSONString(NULL, NULL, NULL);)
2024.11.04 23:20:57.629390 [ 7099 ] {1f6f3874-ca4c-4ff0-b64d-f72ebbdaf681} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toJSONString doesn't match: passed 3, should be 1: In scope SELECT toJSONString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:57.707128 [ 7099 ] {98a7bf9f-9348-4aa8-97f2-8720851f139d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toLowCardinality doesn't match: passed 3, should be 1: In scope SELECT toLowCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toLowCardinality(NULL, NULL, NULL);)
2024.11.04 23:20:57.735845 [ 7099 ] {98a7bf9f-9348-4aa8-97f2-8720851f139d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toLowCardinality doesn't match: passed 3, should be 1: In scope SELECT toLowCardinality(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:57.864787 [ 4556 ] {} <Error> ZooKeeperClient: Code: 999. Coordination::Exception: Session expired (fault injected on recv). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [41]>(char const (&) [41], Coordination::Error) @ 0x0000000039c4f4d1
3. ./src/Common/ZooKeeper/IKeeper.h:509: Coordination::Exception Coordination::Exception::fromMessage<char const (&) [41]>(Coordination::Error, char const (&) [41]) @ 0x000000003c16f344
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1611: Coordination::ZooKeeper::maybeInjectRecvFault() @ 0x000000003c15a10f
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c156cd0
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
8. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
10. ? @ 0x00007fe94d0adac3
11. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:57.865701 [ 3126 ] {} <Warning> test_13.wikistat1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.865805 [ 3138 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.865925 [ 3149 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.866059 [ 3141 ] {} <Warning> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.866295 [ 3119 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.866459 [ 3155 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.866854 [ 3156 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.866898 [ 3152 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:20:57.867655 [ 3115 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_03217_system_tables_replica/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:57.868303 [ 3120 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:57.868348 [ 3545 ] {} <Error> test_13.wikistat1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/tables/test_13/02494_zero_copy_and_projection/async_blocks. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03acc9
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x000000003c039e0d
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:514: DB::ReplicatedMergeTreeCleanupThread::getBlocksSortedByTime(String const&, zkutil::ZooKeeper&, std::vector<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat, std::allocator<DB::ReplicatedMergeTreeCleanupThread::NodeWithStat>>&, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039aaa97a
7. ./contrib/llvm-project/libcxx/include/vector:543: DB::ReplicatedMergeTreeCleanupThread::clearOldBlocks(String const&, unsigned long, unsigned long, std::map<String, std::pair<long, int>, std::less<String>, std::allocator<std::pair<String const, std::pair<long, int>>>>&) @ 0x0000000039a96541
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a81295
9. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
11. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
13. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:57.868549 [ 3132 ] {} <Error> test_13.test_03217_system_tables_replica_2 (1c35257c-fea1-421d-b94e-ca1461972e13): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000003c0dc8b1
3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000003c15b570
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1401: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000003c165c9f
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003c039542
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:218: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c038686
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03ab67
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:254: zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x000000003c03a86a
9. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000039b4c6b1
10. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3573: DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0x000000003820893d
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a59f
12. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
14. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
15. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:57.868633 [ 3148 ] {} <Error> test_13.alter_compression_codec2 (ae6eb19f-238a-4f7a-ba11-65af488d5d63): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/replicas/2_r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:5578: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x00000000380482b1
4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:3522: DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x0000000038207d00
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003841a55f
6. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
8. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007fe94d0adac3
12. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:20:57.876029 [ 7099 ] {016cb47b-684e-4d43-9fb8-6ccd59db2d59} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDay(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toModifiedJulianDay(NULL, NULL, NULL);)
2024.11.04 23:20:57.878508 [ 7099 ] {016cb47b-684e-4d43-9fb8-6ccd59db2d59} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDay(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:57.896129 [ 7099 ] {82e78f1d-3a04-4c39-b680-8347e302fc41} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDayOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toModifiedJulianDayOrNull(NULL, NULL, NULL);)
2024.11.04 23:20:57.898250 [ 7099 ] {82e78f1d-3a04-4c39-b680-8347e302fc41} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDayOrNull(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:58.085181 [ 7099 ] {2fc4efca-975c-4972-8c90-56b9b957eeb8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toNullable doesn't match: passed 3, should be 1: In scope SELECT toNullable(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toNullable(NULL, NULL, NULL);)
2024.11.04 23:20:58.087228 [ 7099 ] {2fc4efca-975c-4972-8c90-56b9b957eeb8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toNullable doesn't match: passed 3, should be 1: In scope SELECT toNullable(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:59.662761 [ 7099 ] {158a3206-d49c-424e-8cbd-0129be6449a3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toStringCutToZero doesn't match: passed 3, should be 1: In scope SELECT toStringCutToZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toStringCutToZero(NULL, NULL, NULL);)
2024.11.04 23:20:59.664885 [ 7099 ] {158a3206-d49c-424e-8cbd-0129be6449a3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStringCutToZero doesn't match: passed 3, should be 1: In scope SELECT toStringCutToZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:59.769682 [ 3735 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:59.769805 [ 3735 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:20:59.776648 [ 7099 ] {859feeb8-2e7e-478f-b513-d7c416008814} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimeZone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toTimeZone(NULL, NULL, NULL);)
2024.11.04 23:20:59.778861 [ 7099 ] {859feeb8-2e7e-478f-b513-d7c416008814} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimeZone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:59.893326 [ 7099 ] {0023e0b4-6750-4f45-aee5-8303a6003c09} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimezone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toTimezone(NULL, NULL, NULL);)
2024.11.04 23:20:59.895577 [ 7099 ] {0023e0b4-6750-4f45-aee5-8303a6003c09} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimezone(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:59.916959 [ 7099 ] {036e7241-de4c-432e-b918-043d5e906c88} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toTypeName doesn't match: passed 3, should be 1: In scope SELECT toTypeName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toTypeName(NULL, NULL, NULL);)
2024.11.04 23:20:59.919261 [ 7099 ] {036e7241-de4c-432e-b918-043d5e906c88} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toTypeName doesn't match: passed 3, should be 1: In scope SELECT toTypeName(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:20:59.976004 [ 7099 ] {7e74dbdd-17a9-4f28-b1a3-bced1a51e6f0} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt128OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt128OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUInt128OrDefault(NULL, NULL, NULL);)
2024.11.04 23:20:59.979331 [ 7099 ] {7e74dbdd-17a9-4f28-b1a3-bced1a51e6f0} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt128OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt128OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:00.150359 [ 7099 ] {3b9acff5-4bcc-4e68-b13b-bba1f779efd1} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt16OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt16OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUInt16OrDefault(NULL, NULL, NULL);)
2024.11.04 23:21:00.153175 [ 7099 ] {3b9acff5-4bcc-4e68-b13b-bba1f779efd1} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt16OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt16OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:00.298297 [ 7099 ] {0a6b5872-9b7a-4090-8235-51fc6490e9c5} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt256OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt256OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUInt256OrDefault(NULL, NULL, NULL);)
2024.11.04 23:21:00.391722 [ 7099 ] {0a6b5872-9b7a-4090-8235-51fc6490e9c5} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt256OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt256OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:00.622762 [ 7099 ] {eb116b4f-587a-4f99-82d4-e035fe3dc650} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUInt32OrDefault(NULL, NULL, NULL);)
2024.11.04 23:21:00.624676 [ 7099 ] {eb116b4f-587a-4f99-82d4-e035fe3dc650} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt32OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt32OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:00.790233 [ 7099 ] {421bb976-3fa0-43cf-8aab-fc1b7d125384} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt64OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt64OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUInt64OrDefault(NULL, NULL, NULL);)
2024.11.04 23:21:00.792090 [ 7099 ] {421bb976-3fa0-43cf-8aab-fc1b7d125384} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt64OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt64OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:00.908999 [ 7099 ] {d744ab8e-06d3-4f0d-93e0-56211cb119e0} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt8OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt8OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUInt8OrDefault(NULL, NULL, NULL);)
2024.11.04 23:21:00.910853 [ 7099 ] {d744ab8e-06d3-4f0d-93e0-56211cb119e0} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toUInt8OrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUInt8OrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:00.976663 [ 7099 ] {5d790bdc-7038-4964-9f09-b67b0b826e65} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT toUTCTimestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUTCTimestamp(NULL, NULL, NULL);)
2024.11.04 23:21:00.978427 [ 7099 ] {5d790bdc-7038-4964-9f09-b67b0b826e65} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT toUTCTimestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.077196 [ 7099 ] {498d9e88-664c-4c75-9844-c08f5c0f5005} <Error> executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for function toUUIDOrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUUIDOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUUIDOrDefault(NULL, NULL, NULL);)
2024.11.04 23:21:01.079315 [ 7099 ] {498d9e88-664c-4c75-9844-c08f5c0f5005} <Error> TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for function toUUIDOrDefault provided 3 (Nullable(Nothing), Nullable(Nothing), Nullable(Nothing)), expected 1 to 2 ('Value' : , ['default_value' : ]): In scope SELECT toUUIDOrDefault(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.130756 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/srg/igwqngjrhykpgwyjgcpthyeidzhqz, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3d5bf9b5-ba29-4433-a01a-63a6608689a8, upload has finished false.
2024.11.04 23:21:01.130915 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/srg/igwqngjrhykpgwyjgcpthyeidzhqz, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3d5bf9b5-ba29-4433-a01a-63a6608689a8, upload has finished false
2024.11.04 23:21:01.138640 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/srg/igwqngjrhykpgwyjgcpthyeidzhqz, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 3d5bf9b5-ba29-4433-a01a-63a6608689a8, upload has finished false
2024.11.04 23:21:01.141921 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:01.142728 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 15 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:01.286281 [ 7099 ] {f67af46f-97f8-4fcf-9c83-70cc6743248b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Micro doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Micro(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUnixTimestamp64Micro(NULL, NULL, NULL);)
2024.11.04 23:21:01.288063 [ 7099 ] {f67af46f-97f8-4fcf-9c83-70cc6743248b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Micro doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Micro(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.301587 [ 7099 ] {28303f2a-b936-4589-a58a-c1040524ea3e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Milli doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Milli(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUnixTimestamp64Milli(NULL, NULL, NULL);)
2024.11.04 23:21:01.302941 [ 7099 ] {28303f2a-b936-4589-a58a-c1040524ea3e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Milli doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Milli(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.315756 [ 7099 ] {3cb4f5da-e91d-44ce-a453-a85aa4917dd8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Nano doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Nano(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toUnixTimestamp64Nano(NULL, NULL, NULL);)
2024.11.04 23:21:01.317152 [ 7099 ] {3cb4f5da-e91d-44ce-a453-a85aa4917dd8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Nano doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Nano(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.327972 [ 7099 ] {5a65a077-8469-4ff8-b0cf-97b914c8330b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT toValidUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT toValidUTF8(NULL, NULL, NULL);)
2024.11.04 23:21:01.329465 [ 7099 ] {5a65a077-8469-4ff8-b0cf-97b914c8330b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT toValidUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.719136 [ 7099 ] {7bbd9240-489f-4bde-ad49-5a872915be85} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT to_utc_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT to_utc_timestamp(NULL, NULL, NULL);)
2024.11.04 23:21:01.722777 [ 7099 ] {7bbd9240-489f-4bde-ad49-5a872915be85} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT to_utc_timestamp(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.741414 [ 7099 ] {3c4030d9-27d4-417d-9238-34d9a681fe4f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT today(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT today(NULL, NULL, NULL);)
2024.11.04 23:21:01.743622 [ 7099 ] {3c4030d9-27d4-417d-9238-34d9a681fe4f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT today(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:01.760909 [ 7099 ] {7d850fe5-48f9-434b-b0a6-c9b51704d837} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tokens doesn't match: passed 3, should be 1: In scope SELECT tokens(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tokens(NULL, NULL, NULL);)
2024.11.04 23:21:01.762837 [ 7099 ] {7d850fe5-48f9-434b-b0a6-c9b51704d837} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tokens doesn't match: passed 3, should be 1: In scope SELECT tokens(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.116173 [ 7099 ] {c9f62db8-343f-43aa-8e27-a1a4ddb68db5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function topLevelDomain doesn't match: passed 3, should be 1: In scope SELECT topLevelDomain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT topLevelDomain(NULL, NULL, NULL);)
2024.11.04 23:21:02.118847 [ 7099 ] {c9f62db8-343f-43aa-8e27-a1a4ddb68db5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function topLevelDomain doesn't match: passed 3, should be 1: In scope SELECT topLevelDomain(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.148794 [ 7099 ] {6ae49e9a-9e27-4ff9-9edf-af350ee97cfe} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function topLevelDomainRFC doesn't match: passed 3, should be 1: In scope SELECT topLevelDomainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT topLevelDomainRFC(NULL, NULL, NULL);)
2024.11.04 23:21:02.150997 [ 7099 ] {6ae49e9a-9e27-4ff9-9edf-af350ee97cfe} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function topLevelDomainRFC doesn't match: passed 3, should be 1: In scope SELECT topLevelDomainRFC(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.166566 [ 7099 ] {2c76d8ab-9d61-42d4-a98f-e17034ca32b1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function transactionID doesn't match: passed 3, should be 0: In scope SELECT transactionID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT transactionID(NULL, NULL, NULL);)
2024.11.04 23:21:02.168596 [ 7099 ] {2c76d8ab-9d61-42d4-a98f-e17034ca32b1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function transactionID doesn't match: passed 3, should be 0: In scope SELECT transactionID(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.184195 [ 7099 ] {387fe40b-c796-4928-a832-7ecf0a3f2988} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function transactionLatestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionLatestSnapshot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT transactionLatestSnapshot(NULL, NULL, NULL);)
2024.11.04 23:21:02.186079 [ 7099 ] {387fe40b-c796-4928-a832-7ecf0a3f2988} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function transactionLatestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionLatestSnapshot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.202610 [ 7099 ] {0dfc923e-d9a9-49f4-9008-8df7a8f4a94b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function transactionOldestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionOldestSnapshot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT transactionOldestSnapshot(NULL, NULL, NULL);)
2024.11.04 23:21:02.204733 [ 7099 ] {0dfc923e-d9a9-49f4-9008-8df7a8f4a94b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function transactionOldestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionOldestSnapshot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.222344 [ 7099 ] {78c8ea1b-f517-4155-a427-048aeb40c2ce} <Error> executeQuery: Code: 43. DB::Exception: Second argument of function transform, must be array of source values to transform from.: In scope SELECT transform(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT transform(NULL, NULL, NULL);)
2024.11.04 23:21:02.224460 [ 7099 ] {78c8ea1b-f517-4155-a427-048aeb40c2ce} <Error> TCPHandler: Code: 43. DB::Exception: Second argument of function transform, must be array of source values to transform from.: In scope SELECT transform(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:21:02.413842 [ 7099 ] {e2969b10-45b3-4060-952c-674adb25a643} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function trimBoth doesn't match: passed 3, should be 1: In scope SELECT trimBoth(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT trimBoth(NULL, NULL, NULL);)
2024.11.04 23:21:02.416037 [ 7099 ] {e2969b10-45b3-4060-952c-674adb25a643} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function trimBoth doesn't match: passed 3, should be 1: In scope SELECT trimBoth(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.431465 [ 7099 ] {1b3c65a7-e254-4a49-bddc-4983a92959cb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function trimLeft doesn't match: passed 3, should be 1: In scope SELECT trimLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT trimLeft(NULL, NULL, NULL);)
2024.11.04 23:21:02.433440 [ 7099 ] {1b3c65a7-e254-4a49-bddc-4983a92959cb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function trimLeft doesn't match: passed 3, should be 1: In scope SELECT trimLeft(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.449462 [ 7099 ] {f22ebd7a-0e9e-4165-b274-fdd2862b1a17} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function trimRight doesn't match: passed 3, should be 1: In scope SELECT trimRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT trimRight(NULL, NULL, NULL);)
2024.11.04 23:21:02.451359 [ 7099 ] {f22ebd7a-0e9e-4165-b274-fdd2862b1a17} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function trimRight doesn't match: passed 3, should be 1: In scope SELECT trimRight(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.527431 [ 7099 ] {54382c6b-6197-4a82-a003-556e1cc6ce77} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tryBase58Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase58Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tryBase58Decode(NULL, NULL, NULL);)
2024.11.04 23:21:02.529814 [ 7099 ] {54382c6b-6197-4a82-a003-556e1cc6ce77} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryBase58Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase58Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.545990 [ 7099 ] {5700290f-bb48-4530-a8c1-8e526c6082a7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tryBase64Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase64Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tryBase64Decode(NULL, NULL, NULL);)
2024.11.04 23:21:02.548144 [ 7099 ] {5700290f-bb48-4530-a8c1-8e526c6082a7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryBase64Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase64Decode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.631058 [ 7099 ] {41fbff97-68a5-463a-a5ff-ce4adbdec7c3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tryIdnaEncode doesn't match: passed 3, should be 1: In scope SELECT tryIdnaEncode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tryIdnaEncode(NULL, NULL, NULL);)
2024.11.04 23:21:02.632971 [ 7099 ] {41fbff97-68a5-463a-a5ff-ce4adbdec7c3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryIdnaEncode doesn't match: passed 3, should be 1: In scope SELECT tryIdnaEncode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.646299 [ 7099 ] {30afc678-8f27-4b63-a04b-13f5fb74d0a4} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tryPunycodeDecode doesn't match: passed 3, should be 1: In scope SELECT tryPunycodeDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tryPunycodeDecode(NULL, NULL, NULL);)
2024.11.04 23:21:02.647835 [ 7099 ] {30afc678-8f27-4b63-a04b-13f5fb74d0a4} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryPunycodeDecode doesn't match: passed 3, should be 1: In scope SELECT tryPunycodeDecode(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.929186 [ 7099 ] {9ba3ad61-b22f-4466-9ce9-7242da76d321} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleDivide doesn't match: passed 3, should be 2: In scope SELECT tupleDivide(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleDivide(NULL, NULL, NULL);)
2024.11.04 23:21:02.930999 [ 7099 ] {9ba3ad61-b22f-4466-9ce9-7242da76d321} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleDivide doesn't match: passed 3, should be 2: In scope SELECT tupleDivide(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.945593 [ 7099 ] {cb070599-9b44-448a-b0e1-8f4fd0f8c27f} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleDivideByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleDivideByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleDivideByNumber(NULL, NULL, NULL);)
2024.11.04 23:21:02.947394 [ 7099 ] {cb070599-9b44-448a-b0e1-8f4fd0f8c27f} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleDivideByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleDivideByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.961147 [ 7099 ] {c07af263-5915-4055-9d68-830b9e2bfe98} <Error> executeQuery: Code: 43. DB::Exception: First argument for function tupleElement must be tuple or array of tuple. Actual Nullable(Nothing): In scope SELECT tupleElement(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleElement(NULL, NULL, NULL);)
2024.11.04 23:21:02.962749 [ 7099 ] {c07af263-5915-4055-9d68-830b9e2bfe98} <Error> TCPHandler: Code: 43. DB::Exception: First argument for function tupleElement must be tuple or array of tuple. Actual Nullable(Nothing): In scope SELECT tupleElement(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:21:02.975625 [ 7099 ] {48ff7afd-98ce-43c7-935f-62abe46291b8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleHammingDistance doesn't match: passed 3, should be 2: In scope SELECT tupleHammingDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleHammingDistance(NULL, NULL, NULL);)
2024.11.04 23:21:02.977254 [ 7099 ] {48ff7afd-98ce-43c7-935f-62abe46291b8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleHammingDistance doesn't match: passed 3, should be 2: In scope SELECT tupleHammingDistance(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:02.990370 [ 7099 ] {8a9fb677-e504-4eca-8296-29a9468903b5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDiv doesn't match: passed 3, should be 2: In scope SELECT tupleIntDiv(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleIntDiv(NULL, NULL, NULL);)
2024.11.04 23:21:02.991937 [ 7099 ] {8a9fb677-e504-4eca-8296-29a9468903b5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDiv doesn't match: passed 3, should be 2: In scope SELECT tupleIntDiv(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.005145 [ 7099 ] {d7157a21-d7a0-4759-a707-af18ba76ae79} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDivByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleIntDivByNumber(NULL, NULL, NULL);)
2024.11.04 23:21:03.007017 [ 7099 ] {d7157a21-d7a0-4759-a707-af18ba76ae79} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDivByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.020023 [ 7099 ] {b7444501-f0cc-4c05-84a0-599b45aef3e5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZero doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleIntDivOrZero(NULL, NULL, NULL);)
2024.11.04 23:21:03.021685 [ 7099 ] {b7444501-f0cc-4c05-84a0-599b45aef3e5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZero doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZero(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.035489 [ 7099 ] {d0b96336-307b-4ad1-ba45-597a663239e6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZeroByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZeroByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleIntDivOrZeroByNumber(NULL, NULL, NULL);)
2024.11.04 23:21:03.037140 [ 7099 ] {d0b96336-307b-4ad1-ba45-597a663239e6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZeroByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZeroByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.052093 [ 7099 ] {e1827685-3e4b-42a1-959c-86da26d232e5} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT tupleMinus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleMinus(NULL, NULL, NULL);)
2024.11.04 23:21:03.054495 [ 7099 ] {e1827685-3e4b-42a1-959c-86da26d232e5} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT tupleMinus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.068104 [ 7099 ] {c800681e-ceb4-4912-bff4-8c1710f0b5da} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleModulo doesn't match: passed 3, should be 2: In scope SELECT tupleModulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleModulo(NULL, NULL, NULL);)
2024.11.04 23:21:03.069706 [ 7099 ] {c800681e-ceb4-4912-bff4-8c1710f0b5da} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleModulo doesn't match: passed 3, should be 2: In scope SELECT tupleModulo(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.082440 [ 7099 ] {3d217345-eb2a-45ac-9a2c-7c45061c4176} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleModuloByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleModuloByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleModuloByNumber(NULL, NULL, NULL);)
2024.11.04 23:21:03.084031 [ 7099 ] {3d217345-eb2a-45ac-9a2c-7c45061c4176} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleModuloByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleModuloByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.096913 [ 7099 ] {812dccb1-d228-4ce1-8103-1352c8bc9d20} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMultiply doesn't match: passed 3, should be 2: In scope SELECT tupleMultiply(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleMultiply(NULL, NULL, NULL);)
2024.11.04 23:21:03.098615 [ 7099 ] {812dccb1-d228-4ce1-8103-1352c8bc9d20} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMultiply doesn't match: passed 3, should be 2: In scope SELECT tupleMultiply(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.107392 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:03.112324 [ 7099 ] {742a9d8e-8cf4-4c81-927a-264adc72ae37} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMultiplyByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleMultiplyByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleMultiplyByNumber(NULL, NULL, NULL);)
2024.11.04 23:21:03.114083 [ 7099 ] {742a9d8e-8cf4-4c81-927a-264adc72ae37} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMultiplyByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleMultiplyByNumber(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.127309 [ 7099 ] {0e16641b-c983-46e9-a04b-73070865b13d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleNegate doesn't match: passed 3, should be 1: In scope SELECT tupleNegate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleNegate(NULL, NULL, NULL);)
2024.11.04 23:21:03.128943 [ 7099 ] {0e16641b-c983-46e9-a04b-73070865b13d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleNegate doesn't match: passed 3, should be 1: In scope SELECT tupleNegate(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.142168 [ 7099 ] {5b390884-0142-4a3f-b6da-3838876402ca} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT tuplePlus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tuplePlus(NULL, NULL, NULL);)
2024.11.04 23:21:03.143761 [ 7099 ] {5b390884-0142-4a3f-b6da-3838876402ca} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT tuplePlus(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.248697 [ 7099 ] {073b0c44-f225-4e37-8f17-990ab415dbc7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleToNameValuePairs doesn't match: passed 3, should be 1: In scope SELECT tupleToNameValuePairs(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT tupleToNameValuePairs(NULL, NULL, NULL);)
2024.11.04 23:21:03.250736 [ 7099 ] {073b0c44-f225-4e37-8f17-990ab415dbc7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleToNameValuePairs doesn't match: passed 3, should be 1: In scope SELECT tupleToNameValuePairs(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.267521 [ 7099 ] {db08ab49-53d2-4841-8eb5-ca1c8a493ebf} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT ucase(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT ucase(NULL, NULL, NULL);)
2024.11.04 23:21:03.269731 [ 7099 ] {db08ab49-53d2-4841-8eb5-ca1c8a493ebf} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT ucase(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.339120 [ 7099 ] {a9216406-a0ff-4418-b4a2-358a0da2e448} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function unbin doesn't match: passed 3, should be 1: In scope SELECT unbin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT unbin(NULL, NULL, NULL);)
2024.11.04 23:21:03.341300 [ 7099 ] {a9216406-a0ff-4418-b4a2-358a0da2e448} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function unbin doesn't match: passed 3, should be 1: In scope SELECT unbin(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.356161 [ 7099 ] {2cd90788-af7e-4f3f-99f3-f5b3ea9f22c3} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function unhex doesn't match: passed 3, should be 1: In scope SELECT unhex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT unhex(NULL, NULL, NULL);)
2024.11.04 23:21:03.358097 [ 7099 ] {2cd90788-af7e-4f3f-99f3-f5b3ea9f22c3} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function unhex doesn't match: passed 3, should be 1: In scope SELECT unhex(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:03.394954 [ 7099 ] {912d898e-9d85-4e70-84a5-8a0fc52e77df} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniq(NULL, NULL, NULL);)
2024.11.04 23:21:03.398662 [ 7099 ] {912d898e-9d85-4e70-84a5-8a0fc52e77df} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:03.562119 [ 7099 ] {862e5964-fdb0-4df0-9c75-eb17b101df7a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniqCombined(NULL, NULL, NULL);)
2024.11.04 23:21:03.566895 [ 7099 ] {862e5964-fdb0-4df0-9c75-eb17b101df7a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:03.690973 [ 7099 ] {6d296c77-bf7b-4ea3-bec2-6c6083a8ea7b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniqCombined64(NULL, NULL, NULL);)
2024.11.04 23:21:03.696488 [ 7099 ] {6d296c77-bf7b-4ea3-bec2-6c6083a8ea7b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:03.798305 [ 7099 ] {7e3abbb3-521d-4f78-8ed3-937129bde2e1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniqExact(NULL, NULL, NULL);)
2024.11.04 23:21:03.803011 [ 7099 ] {7e3abbb3-521d-4f78-8ed3-937129bde2e1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:03.947748 [ 7099 ] {5ec17ee9-e0fd-4c1a-8005-cbf44f3aacac} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniqTheta(NULL, NULL, NULL);)
2024.11.04 23:21:03.953114 [ 7099 ] {5ec17ee9-e0fd-4c1a-8005-cbf44f3aacac} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:03.969215 [ 7099 ] {da486c30-c5c4-467e-9b86-01403818602b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function uniqThetaIntersect doesn't match: passed 3, should be 2: In scope SELECT uniqThetaIntersect(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniqThetaIntersect(NULL, NULL, NULL);)
2024.11.04 23:21:03.971227 [ 7099 ] {da486c30-c5c4-467e-9b86-01403818602b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function uniqThetaIntersect doesn't match: passed 3, should be 2: In scope SELECT uniqThetaIntersect(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.012897 [ 7099 ] {76af2397-4b5a-4036-8fd0-a53c7bc1a6e8} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function uniqThetaNot doesn't match: passed 3, should be 2: In scope SELECT uniqThetaNot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniqThetaNot(NULL, NULL, NULL);)
2024.11.04 23:21:04.015097 [ 7099 ] {76af2397-4b5a-4036-8fd0-a53c7bc1a6e8} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function uniqThetaNot doesn't match: passed 3, should be 2: In scope SELECT uniqThetaNot(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.031464 [ 7099 ] {5acbb28a-28c8-4e35-83bc-24f1d73bb3df} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function uniqThetaUnion doesn't match: passed 3, should be 2: In scope SELECT uniqThetaUnion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uniqThetaUnion(NULL, NULL, NULL);)
2024.11.04 23:21:04.033677 [ 7099 ] {5acbb28a-28c8-4e35-83bc-24f1d73bb3df} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function uniqThetaUnion doesn't match: passed 3, should be 2: In scope SELECT uniqThetaUnion(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.097612 [ 7099 ] {6ef804ec-91f9-4ded-b435-0ee118c16fdd} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT upper(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT upper(NULL, NULL, NULL);)
2024.11.04 23:21:04.099879 [ 7099 ] {6ef804ec-91f9-4ded-b435-0ee118c16fdd} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT upper(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.115534 [ 7099 ] {fa18dce9-1c71-4bf1-9423-9e1773ce2bd1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function upperUTF8 doesn't match: passed 3, should be 1: In scope SELECT upperUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT upperUTF8(NULL, NULL, NULL);)
2024.11.04 23:21:04.117481 [ 7099 ] {fa18dce9-1c71-4bf1-9423-9e1773ce2bd1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function upperUTF8 doesn't match: passed 3, should be 1: In scope SELECT upperUTF8(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.132448 [ 7099 ] {7e4a6f65-b8d3-44ed-ad86-de730919f9fb} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function uptime doesn't match: passed 3, should be 0: In scope SELECT uptime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT uptime(NULL, NULL, NULL);)
2024.11.04 23:21:04.134496 [ 7099 ] {7e4a6f65-b8d3-44ed-ad86-de730919f9fb} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function uptime doesn't match: passed 3, should be 0: In scope SELECT uptime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.241735 [ 7099 ] {59778902-197f-4fbf-b17b-e657ad68f54b} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT user(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT user(NULL, NULL, NULL);)
2024.11.04 23:21:04.243915 [ 7099 ] {59778902-197f-4fbf-b17b-e657ad68f54b} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT user(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.495278 [ 7099 ] {4458f514-aa69-4fc1-b6bf-9cc5b3422c4d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT varPopStable(NULL, NULL, NULL);)
2024.11.04 23:21:04.499566 [ 7099 ] {4458f514-aa69-4fc1-b6bf-9cc5b3422c4d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:04.735592 [ 7099 ] {52cf21b2-d4b9-4638-9ea6-c8542f2ef6de} <Error> executeQuery: Code: 43. DB::Exception: First argument for function variantElement must be Variant or Array of Variant. Actual Nullable(Nothing): In scope SELECT variantElement(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT variantElement(NULL, NULL, NULL);)
2024.11.04 23:21:04.737863 [ 7099 ] {52cf21b2-d4b9-4638-9ea6-c8542f2ef6de} <Error> TCPHandler: Code: 43. DB::Exception: First argument for function variantElement must be Variant or Array of Variant. Actual Nullable(Nothing): In scope SELECT variantElement(NULL, NULL, NULL). (ILLEGAL_TYPE_OF_ARGUMENT)
2024.11.04 23:21:04.753407 [ 7099 ] {01c1e817-e2b3-4db6-a493-e2ecce902bd1} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function variantType doesn't match: passed 3, should be 1: In scope SELECT variantType(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT variantType(NULL, NULL, NULL);)
2024.11.04 23:21:04.755223 [ 7099 ] {01c1e817-e2b3-4db6-a493-e2ecce902bd1} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function variantType doesn't match: passed 3, should be 1: In scope SELECT variantType(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.857295 [ 7099 ] {1a62adb7-42e0-49c4-984d-2d6d135d565e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT vectorDifference(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT vectorDifference(NULL, NULL, NULL);)
2024.11.04 23:21:04.859280 [ 7099 ] {1a62adb7-42e0-49c4-984d-2d6d135d565e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT vectorDifference(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.911173 [ 7099 ] {41a395dd-b6ea-4e89-9b8f-63ce37af0f30} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT vectorSum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT vectorSum(NULL, NULL, NULL);)
2024.11.04 23:21:04.913440 [ 7099 ] {41a395dd-b6ea-4e89-9b8f-63ce37af0f30} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT vectorSum(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:04.930476 [ 7099 ] {ac39b742-00f5-4802-964e-872fd6dc93f6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function version doesn't match: passed 3, should be 0: In scope SELECT version(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT version(NULL, NULL, NULL);)
2024.11.04 23:21:04.932528 [ 7099 ] {ac39b742-00f5-4802-964e-872fd6dc93f6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function version doesn't match: passed 3, should be 0: In scope SELECT version(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.006550 [ 7099 ] {abddb186-4d80-4d27-88ca-a5af3c009682} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function visibleWidth doesn't match: passed 3, should be 1: In scope SELECT visibleWidth(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visibleWidth(NULL, NULL, NULL);)
2024.11.04 23:21:05.009124 [ 7099 ] {abddb186-4d80-4d27-88ca-a5af3c009682} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function visibleWidth doesn't match: passed 3, should be 1: In scope SELECT visibleWidth(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.030314 [ 7099 ] {100399d4-89ce-4d03-9c03-d9e5c1332978} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractBool(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visitParamExtractBool(NULL, NULL, NULL);)
2024.11.04 23:21:05.032428 [ 7099 ] {100399d4-89ce-4d03-9c03-d9e5c1332978} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractBool(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.034657 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:05.041233 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_106_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:05.046945 [ 7099 ] {12f53854-6f07-42c3-bd21-880cea64c70e} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractFloat(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visitParamExtractFloat(NULL, NULL, NULL);)
2024.11.04 23:21:05.048918 [ 7099 ] {12f53854-6f07-42c3-bd21-880cea64c70e} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractFloat(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.061370 [ 7099 ] {b985733a-fedb-46b4-a7e6-9ed8060d94c6} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visitParamExtractInt(NULL, NULL, NULL);)
2024.11.04 23:21:05.062853 [ 7099 ] {b985733a-fedb-46b4-a7e6-9ed8060d94c6} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.074100 [ 7099 ] {1cae7d86-67e9-4472-97ba-f6726cdf06fa} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractRaw(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visitParamExtractRaw(NULL, NULL, NULL);)
2024.11.04 23:21:05.075416 [ 7099 ] {1cae7d86-67e9-4472-97ba-f6726cdf06fa} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractRaw(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.086249 [ 7099 ] {591106ed-bb93-4c10-9264-ff49fd4b1c88} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visitParamExtractString(NULL, NULL, NULL);)
2024.11.04 23:21:05.087665 [ 7099 ] {591106ed-bb93-4c10-9264-ff49fd4b1c88} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractString(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.098388 [ 7099 ] {199ae159-6259-4811-9aeb-20fc04c2c009} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractUInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visitParamExtractUInt(NULL, NULL, NULL);)
2024.11.04 23:21:05.099905 [ 7099 ] {199ae159-6259-4811-9aeb-20fc04c2c009} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractUInt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.150615 [ 7099 ] {56b39618-18bf-43f7-a2b7-5b97fdc31061} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT visitParamHas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT visitParamHas(NULL, NULL, NULL);)
2024.11.04 23:21:05.152889 [ 7099 ] {56b39618-18bf-43f7-a2b7-5b97fdc31061} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT visitParamHas(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.308747 [ 7099 ] {c4bee779-5c5a-4016-980d-c547cddb433a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT welchTTest(NULL, NULL, NULL);)
2024.11.04 23:21:05.312800 [ 7099 ] {c4bee779-5c5a-4016-980d-c547cddb433a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:05.327819 [ 7099 ] {bca06f87-131d-4721-9a27-8252c17b15d7} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT widthBucket(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT widthBucket(NULL, NULL, NULL);)
2024.11.04 23:21:05.329755 [ 7099 ] {bca06f87-131d-4721-9a27-8252c17b15d7} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT widthBucket(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.344632 [ 7099 ] {851b55a8-1f35-4139-9e46-09a73168d18d} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT width_bucket(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT width_bucket(NULL, NULL, NULL);)
2024.11.04 23:21:05.346652 [ 7099 ] {851b55a8-1f35-4139-9e46-09a73168d18d} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT width_bucket(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:05.386348 [ 7099 ] {0b60b62a-b545-4fdb-9db0-7f31984d7b3c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT windowFunnel(NULL, NULL, NULL);)
2024.11.04 23:21:05.390589 [ 7099 ] {0b60b62a-b545-4fdb-9db0-7f31984d7b3c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK)
2024.11.04 23:21:05.482865 [ 7099 ] {7c1c6ca9-d6bb-4f89-95a6-d50847569f66} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function wkt doesn't match: passed 3, should be 1: In scope SELECT wkt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT wkt(NULL, NULL, NULL);)
2024.11.04 23:21:05.485103 [ 7099 ] {7c1c6ca9-d6bb-4f89-95a6-d50847569f66} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function wkt doesn't match: passed 3, should be 1: In scope SELECT wkt(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:06.390241 [ 7099 ] {f89a1c9f-c71f-4165-9d62-ebd9137421ad} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT yandexConsistentHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT yandexConsistentHash(NULL, NULL, NULL);)
2024.11.04 23:21:06.392750 [ 7099 ] {f89a1c9f-c71f-4165-9d62-ebd9137421ad} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT yandexConsistentHash(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:06.429600 [ 7099 ] {79d4a5da-3df6-4738-9076-d65ada85e844} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function yesterday doesn't match: passed 3, should be 0: In scope SELECT yesterday(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT yesterday(NULL, NULL, NULL);)
2024.11.04 23:21:06.432702 [ 7099 ] {79d4a5da-3df6-4738-9076-d65ada85e844} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function yesterday doesn't match: passed 3, should be 0: In scope SELECT yesterday(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:06.443958 [ 7099 ] {0a192f07-53ef-4b0f-adcc-e0be12b950fc} <Error> executeQuery: Code: 42. DB::Exception: Number of arguments for function zookeeperSessionUptime doesn't match: passed 3, should be 0: In scope SELECT zookeeperSessionUptime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:51274) (comment: 00534_functions_bad_arguments11.sh) (in query: SELECT zookeeperSessionUptime(NULL, NULL, NULL);)
2024.11.04 23:21:06.445567 [ 7099 ] {0a192f07-53ef-4b0f-adcc-e0be12b950fc} <Error> TCPHandler: Code: 42. DB::Exception: Number of arguments for function zookeeperSessionUptime doesn't match: passed 3, should be 0: In scope SELECT zookeeperSessionUptime(NULL, NULL, NULL). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH)
2024.11.04 23:21:06.930035 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:08.556054 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:08.556330 [ 3948 ] {686c2ad1-b34d-4115-a0f9-8113bda4b651} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36204) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:08.562429 [ 3948 ] {686c2ad1-b34d-4115-a0f9-8113bda4b651} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:08.564068 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_109_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:08.625286 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/vaj/fpkblckqyntzwfchchazmrvfhqktf, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65985f95-785f-45d1-a938-bc26d2f6f8f4, upload has finished false.
2024.11.04 23:21:08.625503 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/vaj/fpkblckqyntzwfchchazmrvfhqktf, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65985f95-785f-45d1-a938-bc26d2f6f8f4, upload has finished false
2024.11.04 23:21:08.691713 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/vaj/fpkblckqyntzwfchchazmrvfhqktf, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 65985f95-785f-45d1-a938-bc26d2f6f8f4, upload has finished false
2024.11.04 23:21:08.696356 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 45 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:08.780939 [ 3708 ] {d07fe90c-9127-4c42-a897-2be4226fc1eb::1_1_1_0_3} <Error> MutatePlainMergeTreeTask: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:08.781624 [ 3708 ] {d07fe90c-9127-4c42-a897-2be4226fc1eb::1_1_1_0_3} <Error> virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:08.782492 [ 2935 ] {5797ebf5-7445-46da-bedc-ff7e150bf9dd} <Error> executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_3.txt' with part '1_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36208) (comment: 03221_mutation_analyzer_skip_part.sql) (in query: ALTER TABLE t_mutate_skip_part UPDATE v1 = 1000 WHERE key = 1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:21:08.785503 [ 2935 ] {5797ebf5-7445-46da-bedc-ff7e150bf9dd} <Error> TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_3.txt' with part '1_1_1_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000039d2334d
6. ./contrib/llvm-project/libcxx/include/vector:543: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d59e2
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:21:10.735259 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:10.735390 [ 2935 ] {7e99b403-9759-4a27-a414-c95c531b41cc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36220) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:11.524336 [ 2935 ] {7e99b403-9759-4a27-a414-c95c531b41cc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:11.524588 [ 3948 ] {60ea0aa7-b21a-4fb2-9fd6-592abe89e76c} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n3 which is referenced by materialized view ['01746_merge_tree_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_merge_tree` DROP COLUMN n3;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3313: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000039179341
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:12.486872 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/lui/oqvkayenbwzqspgpkqiifosvacneb, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1c4c2259-b713-456d-aac4-04d951bcd5d6, upload has finished false.
2024.11.04 23:21:12.487042 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/lui/oqvkayenbwzqspgpkqiifosvacneb, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1c4c2259-b713-456d-aac4-04d951bcd5d6, upload has finished false
2024.11.04 23:21:12.493707 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/lui/oqvkayenbwzqspgpkqiifosvacneb, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1c4c2259-b713-456d-aac4-04d951bcd5d6, upload has finished false
2024.11.04 23:21:13.033743 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:13.033764 [ 3948 ] {60ea0aa7-b21a-4fb2-9fd6-592abe89e76c} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n3 which is referenced by materialized view ['01746_merge_tree_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3313: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000039179341
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:13.033977 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_113_23}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:13.034001 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 10 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:13.061607 [ 3948 ] {d50c10c2-c607-4685-b27f-b11e79752ab8} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_merge_tree_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_merge_tree` DROP COLUMN n2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3313: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000039179341
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:13.062719 [ 3948 ] {d50c10c2-c607-4685-b27f-b11e79752ab8} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_merge_tree_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3313: DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000039179341
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:14.212835 [ 3948 ] {18fb601c-7766-413e-9fb6-836dd1fbd89b} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_null_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_null` DROP COLUMN n1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageNull.cpp:58: DB::StorageNull::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037ffc367
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:14.212921 [ 2935 ] {56e1d190-ec92-4298-a6c3-3770d2ea25fe} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56234) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:14.214444 [ 3948 ] {18fb601c-7766-413e-9fb6-836dd1fbd89b} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_null_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageNull.cpp:58: DB::StorageNull::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037ffc367
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:14.217120 [ 2935 ] {56e1d190-ec92-4298-a6c3-3770d2ea25fe} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:14.230249 [ 3948 ] {5ab69ecd-792b-466a-84ab-534463a6ef8b} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_null_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_null` DROP COLUMN n2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageNull.cpp:58: DB::StorageNull::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037ffc367
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:14.231391 [ 3948 ] {5ab69ecd-792b-466a-84ab-534463a6ef8b} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_null_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageNull.cpp:58: DB::StorageNull::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037ffc367
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:14.250890 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_113_23}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:15.030404 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_109_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:15.686048 [ 3948 ] {0b15ae85-1ba7-457e-a94f-d26f8c24911b} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_dist_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_dist` DROP COLUMN n1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageDistributed.cpp:1264: DB::StorageDistributed::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037c2c54f
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:15.686077 [ 2935 ] {5353c9e6-86cf-4390-a2bf-9d33144c8a7c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56238) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:15.687414 [ 3948 ] {0b15ae85-1ba7-457e-a94f-d26f8c24911b} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_dist_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageDistributed.cpp:1264: DB::StorageDistributed::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037c2c54f
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:15.689808 [ 2935 ] {5353c9e6-86cf-4390-a2bf-9d33144c8a7c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:15.704002 [ 3948 ] {db9c7569-1436-4e08-98de-fd707c0bac09} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_dist_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_dist` DROP COLUMN n2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageDistributed.cpp:1264: DB::StorageDistributed::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037c2c54f
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:15.745814 [ 3948 ] {db9c7569-1436-4e08-98de-fd707c0bac09} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_dist_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageDistributed.cpp:1264: DB::StorageDistributed::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037c2c54f
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:15.769969 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_110_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:16.599394 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:17.023517 [ 3948 ] {3466e4f9-0ae7-4e9e-b3a4-2b84d441783c} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_merge_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_merge` DROP COLUMN n1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageMerge.cpp:1442: DB::StorageMerge::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037f8517b
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:17.023561 [ 2935 ] {5b770987-2298-497e-bc8f-9bd7fbf544b7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56242) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:17.024986 [ 3948 ] {3466e4f9-0ae7-4e9e-b3a4-2b84d441783c} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_merge_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageMerge.cpp:1442: DB::StorageMerge::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037f8517b
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:17.026921 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_110_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:17.027119 [ 2935 ] {5b770987-2298-497e-bc8f-9bd7fbf544b7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:17.041598 [ 3948 ] {5a97fe36-cb26-40e0-baff-bd20781e44cd} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_merge_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_merge` DROP COLUMN n2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageMerge.cpp:1442: DB::StorageMerge::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037f8517b
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:17.043058 [ 3948 ] {5a97fe36-cb26-40e0-baff-bd20781e44cd} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_merge_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageMerge.cpp:1442: DB::StorageMerge::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x0000000037f8517b
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:17.411180 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_110_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:18.090567 [ 3098 ] {} <Error> void DB::StorageBuffer::backgroundFlush(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:1050: DB::StorageBuffer::backgroundFlush() @ 0x00000000387e1f16
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000387e667f
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:18.881769 [ 3948 ] {b81e46e6-f925-4686-a999-77b85b6469d5} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_buffer_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_buffer` DROP COLUMN n1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageBuffer.cpp:1103: DB::StorageBuffer::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x00000000387e46bb
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:18.883039 [ 3948 ] {b81e46e6-f925-4686-a999-77b85b6469d5} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n1 which is referenced by materialized view ['01746_buffer_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageBuffer.cpp:1103: DB::StorageBuffer::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x00000000387e46bb
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:18.899025 [ 3948 ] {803ba551-952f-4bdb-a8f1-f61aa2b77a0e} <Error> executeQuery: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_buffer_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: ALTER TABLE `01746_buffer` DROP COLUMN n2;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageBuffer.cpp:1103: DB::StorageBuffer::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x00000000387e46bb
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:18.900179 [ 3948 ] {803ba551-952f-4bdb-a8f1-f61aa2b77a0e} <Error> TCPHandler: Code: 524. DB::Exception: Trying to ALTER DROP column n2 which is referenced by materialized view ['01746_buffer_mv']. (ALTER_OF_COLUMN_IS_FORBIDDEN), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Storages/StorageBuffer.cpp:1103: DB::StorageBuffer::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x00000000387e46bb
4. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d494e
5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:21:19.393178 [ 3948 ] {9d9388b5-5716-4648-a000-24aed49ec23c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36210) (comment: 01746_forbid_drop_column_referenced_by_mv.sql) (in query: -- ok
 ALTER TABLE `01746_buffer` DROP COLUMN n3;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x00000000387d34a8
9. ./contrib/llvm-project/libcxx/include/vector:434: DB::StorageBuffer::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x00000000387e566c
10. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
11. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:21:19.394521 [ 3948 ] {9d9388b5-5716-4648-a000-24aed49ec23c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x00000000387d34a8
9. ./contrib/llvm-project/libcxx/include/vector:434: DB::StorageBuffer::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x00000000387e566c
10. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
11. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
13. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
19. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
20. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
21. ? @ 0x00007fe94d0adac3
22. ? @ 0x00007fe94d13f850

2024.11.04 23:21:19.798669 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/rch/xjljflgmagzpzwfiosphrncblhnwb, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fd1f82fd-1942-4394-a0bd-81e2469ccfc8, upload has finished false.
2024.11.04 23:21:19.798897 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/rch/xjljflgmagzpzwfiosphrncblhnwb, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fd1f82fd-1942-4394-a0bd-81e2469ccfc8, upload has finished false
2024.11.04 23:21:19.836440 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/rch/xjljflgmagzpzwfiosphrncblhnwb, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fd1f82fd-1942-4394-a0bd-81e2469ccfc8, upload has finished false
2024.11.04 23:21:19.840885 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 49 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:20.037373 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:20.197565 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:20.275140 [ 2935 ] {6f8c02f1-c9e6-450a-bc32-1db8ee08c57a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56278) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:20.279465 [ 2935 ] {6f8c02f1-c9e6-450a-bc32-1db8ee08c57a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:20.901613 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_110_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:21.291739 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_111_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:21.886743 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 11 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:24.691225 [ 2935 ] {9c18dd26-eb92-4b6b-a8fb-807e1f871286} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38326) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:24.694976 [ 2935 ] {9c18dd26-eb92-4b6b-a8fb-807e1f871286} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:24.920877 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_111_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:25.915755 [ 2935 ] {5cf49faa-5f41-48f6-bd4b-0fab7e15a466} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38328) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:25.919482 [ 2935 ] {5cf49faa-5f41-48f6-bd4b-0fab7e15a466} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:26.769064 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:21:26.769208 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:21:27.534127 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/vls/njcrzrsuwbaaxnascjzzytykvxalr, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 04e008ed-6e7b-49e8-9996-02cece889cd9, upload has finished false.
2024.11.04 23:21:27.534381 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/vls/njcrzrsuwbaaxnascjzzytykvxalr, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 04e008ed-6e7b-49e8-9996-02cece889cd9, upload has finished false
2024.11.04 23:21:27.562716 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/vls/njcrzrsuwbaaxnascjzzytykvxalr, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 04e008ed-6e7b-49e8-9996-02cece889cd9, upload has finished false
2024.11.04 23:21:27.624071 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 18 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:28.178600 [ 2935 ] {0b83b3b6-9ec1-4fa4-a3b4-42205c2e90da} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:28.182659 [ 2935 ] {0b83b3b6-9ec1-4fa4-a3b4-42205c2e90da} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:28.998877 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_111_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:29.246044 [ 2935 ] {435da73d-bbe9-4e98-bd21-09dfcb865349} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38356) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:29.249812 [ 2935 ] {435da73d-bbe9-4e98-bd21-09dfcb865349} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:29.921252 [ 3948 ] {7f14b8f3-1596-433d-a1d3-00d1a3e7f0f1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56262) (comment: 02867_page_cache.sql) (in query: -- Cold read, should miss cache. (Populating cache on write is not implemented yet.)
 select 'cold read', sum(k) from page_cache_03055;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:29.934799 [ 3948 ] {7f14b8f3-1596-433d-a1d3-00d1a3e7f0f1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:32.573305 [ 3948 ] {6394814e-ff99-4893-99af-2ff6b7aa8f76} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38396) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:32.576604 [ 3948 ] {6394814e-ff99-4893-99af-2ff6b7aa8f76} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:32.659688 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/nkj/qamuylkhrwqsxkiciogillmhhgwgu, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a5023c41-95f3-473a-bf9b-025e0ea38bc3, upload has finished false.
2024.11.04 23:21:32.660035 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/nkj/qamuylkhrwqsxkiciogillmhhgwgu, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a5023c41-95f3-473a-bf9b-025e0ea38bc3, upload has finished false
2024.11.04 23:21:32.673822 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/nkj/qamuylkhrwqsxkiciogillmhhgwgu, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a5023c41-95f3-473a-bf9b-025e0ea38bc3, upload has finished false
2024.11.04 23:21:32.676987 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. ./src/IO/WriteBuffer.h:65: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039408090
17. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&) @ 0x00000000393f910f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f5c43
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:34.504653 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_171_212_8}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:35.566964 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 79 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:35.969138 [ 7099 ] {2be059a2-d9be-4a84-aa77-69b02cb19473} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52516) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:35.972593 [ 7099 ] {2be059a2-d9be-4a84-aa77-69b02cb19473} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:36.385974 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:37.635745 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:38.357727 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 6 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:38.397977 [ 7099 ] {78cfcf30-c2a9-4a11-9c69-4034cb58d0d4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52556) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:38.442287 [ 7099 ] {78cfcf30-c2a9-4a11-9c69-4034cb58d0d4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:38.849920 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_122_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:39.525773 [ 7099 ] {12cfbca9-1637-44ec-8491-a96466981c19} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:52572) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:39.529076 [ 7099 ] {12cfbca9-1637-44ec-8491-a96466981c19} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:40.288019 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_8_8_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:41.057300 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:41.406661 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:42.982420 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:43.957434 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_122_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:44.120027 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:44.165218 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 82 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:46.134177 [ 7099 ] {3ae67c4e-5049-4063-9829-62a3668973ac} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44660) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:46.137497 [ 7099 ] {3ae67c4e-5049-4063-9829-62a3668973ac} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:47.188501 [ 7099 ] {bac137d6-6668-48cd-9202-21b069ca3756} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44670) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:47.192777 [ 7099 ] {bac137d6-6668-48cd-9202-21b069ca3756} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:48.200865 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_122_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=104). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:48.273591 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:49.412063 [ 7099 ] {ad32ba6d-b2d0-482d-8af9-b27ab0497ae9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44690) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:49.415387 [ 7099 ] {ad32ba6d-b2d0-482d-8af9-b27ab0497ae9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:49.755947 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ntl/gtrjyambfdvuolftpzcxcsvlvixgc, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fd5cd057-3a9d-40e0-a7df-6d35cecb96af, upload has finished false.
2024.11.04 23:21:49.756114 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ntl/gtrjyambfdvuolftpzcxcsvlvixgc, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fd5cd057-3a9d-40e0-a7df-6d35cecb96af, upload has finished false
2024.11.04 23:21:49.768527 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ntl/gtrjyambfdvuolftpzcxcsvlvixgc, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fd5cd057-3a9d-40e0-a7df-6d35cecb96af, upload has finished false
2024.11.04 23:21:49.773278 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 7 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:51.248517 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_107_28}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:52.937559 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_122_21}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:53.393641 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:21:53.393811 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:21:53.639516 [ 7099 ] {d788c234-ab41-4ac1-ba91-7e6180a78565} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37004) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:53.643401 [ 7099 ] {d788c234-ab41-4ac1-ba91-7e6180a78565} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:54.063714 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:54.475329 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:55.652321 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/quq/vfkmtyfygcpymfvjtmlxwjgdjfoew, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b9fa79ff-eb21-43bb-a935-34d8f60e3313, upload has finished false.
2024.11.04 23:21:55.652541 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/quq/vfkmtyfygcpymfvjtmlxwjgdjfoew, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b9fa79ff-eb21-43bb-a935-34d8f60e3313, upload has finished false
2024.11.04 23:21:55.659798 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/quq/vfkmtyfygcpymfvjtmlxwjgdjfoew, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b9fa79ff-eb21-43bb-a935-34d8f60e3313, upload has finished false
2024.11.04 23:21:55.664690 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 10 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:55.936144 [ 7099 ] {4bee33ea-950a-4415-86ce-c68ca6afe85d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37020) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:55.942237 [ 7099 ] {4bee33ea-950a-4415-86ce-c68ca6afe85d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:58.066545 [ 3765 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:58.128135 [ 7099 ] {4bcf7f58-5421-4e54-9121-207820bef7bb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37034) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:58.132163 [ 7099 ] {4bcf7f58-5421-4e54-9121-207820bef7bb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:21:58.193396 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 22 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:59.271383 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:21:59.298013 [ 7099 ] {7f87d245-db18-4cdf-a080-7b3326e33549} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37050) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:21:59.302732 [ 7099 ] {7f87d245-db18-4cdf-a080-7b3326e33549} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:02.453318 [ 7099 ] {e379bff1-37ab-4b60-9811-433248c786eb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37056) (comment: 02494_zero_copy_projection_cancel_fetch.sh) (in query: SELECT count() FROM wikistat1 WHERE NOT ignore(*)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:22:02.458479 [ 7099 ] {e379bff1-37ab-4b60-9811-433248c786eb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:177: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b80340a
21. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:37: DB::MergeTreeThreadSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b72dfd4
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
24. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
25. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
27. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
28. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
29. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95

2024.11.04 23:22:02.650644 [ 3948 ] {c2a1166a-482b-4da7-ac04-44127f899838} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37086) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:02.654421 [ 3948 ] {c2a1166a-482b-4da7-ac04-44127f899838} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:02.770787 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:04.513005 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:22:04.513151 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:22:04.828884 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/cvh/pxkofbdvvvwoocsxchkonitdhstwx, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5c4924da-d9ad-487c-b1be-9d1bc7c92186, upload has finished false.
2024.11.04 23:22:04.829155 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/cvh/pxkofbdvvvwoocsxchkonitdhstwx, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5c4924da-d9ad-487c-b1be-9d1bc7c92186, upload has finished false
2024.11.04 23:22:04.836940 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/cvh/pxkofbdvvvwoocsxchkonitdhstwx, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 5c4924da-d9ad-487c-b1be-9d1bc7c92186, upload has finished false
2024.11.04 23:22:04.840589 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 19 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:04.894577 [ 7099 ] {cdca3852-f7e7-40f0-b76e-707c702a9f83} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41036) (comment: 00533_uniq_array.sql) (in query: SELECT uniqArray([0, 1, 1], [0, 1, 1], [0, 1, 1]);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:04.898647 [ 7099 ] {cdca3852-f7e7-40f0-b76e-707c702a9f83} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:05.319047 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_122_25}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:06.703193 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_108_29}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001f17228c
3. ./build_docker/./src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x000000001f16d111
4. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x000000001f05f81f
5. ./build_docker/./src/Common/ThreadFuzzer.cpp:260: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001f1e5b36
6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x0000000039dd0540
7. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccf95
8. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
9. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
11. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
12. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
14. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:06.952934 [ 7099 ] {fce6150c-a787-4a30-a7af-93375c6f7270} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41068) (comment: 02428_batch_nullable_assert.sql) (in query: -- https://github.com/ClickHouse/ClickHouse/issues/41470
 SELECT roundBankers(100), -9223372036854775808, roundBankers(result.2, 256) FROM ( SELECT studentTTest(sample, variant) AS result FROM ( SELECT toFloat64(number) % NULL AS sample, 0 AS variant FROM system.numbers LIMIT 1025 UNION ALL SELECT (toFloat64(number) % 9223372036854775807) + nan AS sample, -9223372036854775808 AS variant FROM system.numbers LIMIT 1024 ) ) FORMAT CSV ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:06.959017 [ 7099 ] {fce6150c-a787-4a30-a7af-93375c6f7270} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:06.966237 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 84 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:07.050961 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:07.209452 [ 7099 ] {84b8616c-3f8f-44e6-bc1c-208a1eaf7266} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41074) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:07.212624 [ 7099 ] {84b8616c-3f8f-44e6-bc1c-208a1eaf7266} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:07.333365 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:08.351934 [ 3948 ] {51d5e68e-1e40-4226-8d57-17a00ea33cf0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41098) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:08.355205 [ 3948 ] {51d5e68e-1e40-4226-8d57-17a00ea33cf0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:10.880320 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988e55c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:10.880482 [ 7099 ] {0f4a2b2c-2ec6-4e68-bd40-5b20737f9510} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41114) (comment: 02006_use_constants_in_with_and_select.sql) (in query: WITH 1 AS max_size SELECT groupArray(max_size)(col) FROM (SELECT col FROM ( SELECT 1 as col UNION ALL SELECT 2 ) ORDER BY col);), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:10.880473 [ 3948 ] {4c20df0e-7e38-47ea-9123-d15423a40b0f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41126) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:10.883679 [ 3948 ] {4c20df0e-7e38-47ea-9123-d15423a40b0f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:10.886044 [ 7099 ] {0f4a2b2c-2ec6-4e68-bd40-5b20737f9510} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:11.840861 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:12.113370 [ 3948 ] {f4aecb4b-17f4-4ac8-8c20-0d430efae989} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41150) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:12.116598 [ 3948 ] {f4aecb4b-17f4-4ac8-8c20-0d430efae989} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:12.714853 [ 7099 ] {3c18056c-b5c5-42de-a3f3-52f9714893d5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:41136) (comment: 00931_low_cardinality_read_with_empty_array.sql) (in query: SELECT * FROM lc_00931 WHERE (key < 100 OR key > 50000) AND NOT has(value, toString(key)) AND length(value) == 1 LIMIT 10 SETTINGS max_block_size = 8192, max_threads = 1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:22:12.718956 [ 7099 ] {3c18056c-b5c5-42de-a3f3-52f9714893d5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:22:12.775758 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=28). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:13.723127 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:13.735595 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:15.329490 [ 3948 ] {959d4553-8d26-43bd-80cd-ef40bddb8f8d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48140) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:15.340114 [ 3948 ] {959d4553-8d26-43bd-80cd-ef40bddb8f8d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:15.546206 [ 7099 ] {380b1a78-55fc-414c-9262-9580ad6bbf73} <Error> executeQuery: Code: 48. DB::Exception: JOIN ON constant supported only with join algorithm 'hash'. (NOT_IMPLEMENTED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48128) (comment: 02001_join_on_const_bs_long.gen.sql) (in query: SELECT count() == 2222 FROM t1 JOIN t2 ON 1 = 1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ab79
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:22:15.550226 [ 7099 ] {380b1a78-55fc-414c-9262-9580ad6bbf73} <Error> TCPHandler: Code: 48. DB::Exception: JOIN ON constant supported only with join algorithm 'hash'. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ab79
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:22:16.393929 [ 3948 ] {5abc4fb5-8d7b-42a1-b1e2-c261db253e01} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48164) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:16.396994 [ 3948 ] {5abc4fb5-8d7b-42a1-b1e2-c261db253e01} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:17.056765 [ 7099 ] {56f9c430-221d-4d43-bcd9-6c8c03d99745} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48152) (comment: 00472_compare_uuid_with_constant_string.sql) (in query: SELECT toUUID('61f0c404-5cb3-11e7-907b-a6006ad3dba2') > '61f0c404-5cb3-11e7-907b-a6006ad3dba1';), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:22:17.059623 [ 7099 ] {56f9c430-221d-4d43-bcd9-6c8c03d99745} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:22:17.235168 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:17.552514 [ 7099 ] {f6ac7f6b-5c3e-41e2-a68a-ed2fe0dcd89e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48170) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:17.556068 [ 7099 ] {f6ac7f6b-5c3e-41e2-a68a-ed2fe0dcd89e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:18.246911 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/jbo/pmrarcsfjumndrbuyfnnjgfqluruc, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1025ff8c-d377-46b3-8561-70b555f2981f, upload has finished false.
2024.11.04 23:22:18.247088 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/jbo/pmrarcsfjumndrbuyfnnjgfqluruc, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1025ff8c-d377-46b3-8561-70b555f2981f, upload has finished false
2024.11.04 23:22:18.303548 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/jbo/pmrarcsfjumndrbuyfnnjgfqluruc, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 1025ff8c-d377-46b3-8561-70b555f2981f, upload has finished false
2024.11.04 23:22:18.310007 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 31 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:19.588844 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:20.003292 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_9_9_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:22.152519 [ 4638 ] {2bcbf432-fedc-4986-8699-11d3e2af3e49} <Warning> DistributedSink: Structure does not match (remote: n Int8 Int8(size = 0), local: n UInt64 UInt64(size = 1)), implicit conversion will be done.
2024.11.04 23:22:22.152520 [ 4734 ] {2bcbf432-fedc-4986-8699-11d3e2af3e49} <Warning> DistributedSink: Structure does not match (remote: n Int8 Int8(size = 0), local: n UInt64 UInt64(size = 1)), implicit conversion will be done.
2024.11.04 23:22:22.851593 [ 7099 ] {2bcbf432-fedc-4986-8699-11d3e2af3e49} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48216) (comment: 01791_dist_INSERT_block_structure_mismatch.sh) (in query: INSERT INTO dist_01683 VALUES ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::onFinish() @ 0x0000000038d580f0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003affde6f
8. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003affcd64
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
13. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:22:22.851727 [ 2935 ] {1a68a520-c526-441a-88a9-6a3afa971f96} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:48228) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:22.856191 [ 2935 ] {1a68a520-c526-441a-88a9-6a3afa971f96} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:22.857096 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_120_24}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:22.859874 [ 7099 ] {2bcbf432-fedc-4986-8699-11d3e2af3e49} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/Distributed/DistributedSink.cpp:0: DB::DistributedSink::onFinish() @ 0x0000000038d580f0
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000003affde6f
8. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003affcd64
10. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
13. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
16. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
20. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
21. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
22. ? @ 0x00007fe94d0adac3
23. ? @ 0x00007fe94d13f850

2024.11.04 23:22:22.890856 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 84 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:23.727239 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_128_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=71). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:23.937505 [ 2935 ] {8d7072e1-45e2-450b-910e-b583608ed5aa} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49732) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:23.940774 [ 2935 ] {8d7072e1-45e2-450b-910e-b583608ed5aa} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:24.013755 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_120_24}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:24.365612 [ 7099 ] {459af713-ce81-40f0-af85-3db029128003} <Error> executeQuery: Code: 47. DB::Exception: Unknown expression identifier 'test_id' in scope SELECT test_id FROM test_table. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49726) (comment: 02337_analyzer_columns_basic.sql) (in query: SELECT test_id FROM test_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String&, String const&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String const&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String const&, String&&, String&&) @ 0x0000000034910cc2
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6420: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347c14e0
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:22:24.366888 [ 7099 ] {459af713-ce81-40f0-af85-3db029128003} <Error> TCPHandler: Code: 47. DB::Exception: Unknown expression identifier 'test_id' in scope SELECT test_id FROM test_table. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String&, String const&, String, String>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String&>::type, std::type_identity<String const&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, char const*&&, String&, String const&, String&&, String&&) @ 0x0000000034910cc2
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6420: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347c14e0
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
13. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
14. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
15. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
17. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
21. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
22. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
23. ? @ 0x00007fe94d0adac3
24. ? @ 0x00007fe94d13f850

2024.11.04 23:22:25.139396 [ 7099 ] {f23ff6be-d306-410d-ba7c-5280f1edb43e} <Error> executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 'test_unknown_table' in scope SELECT test_id FROM test_unknown_table. (UNKNOWN_TABLE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49726) (comment: 02337_analyzer_columns_basic.sql) (in query: SELECT test_id FROM test_unknown_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x000000000a10afd5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6970: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a48b5
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:22:25.139746 [ 2935 ] {2614e897-ca36-4151-a814-69c6119477a4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49744) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:25.141004 [ 7099 ] {f23ff6be-d306-410d-ba7c-5280f1edb43e} <Error> TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 'test_unknown_table' in scope SELECT test_id FROM test_unknown_table. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x000000000a10afd5
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:6970: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347a48b5
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
5. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
6. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
10. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
12. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
14. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
18. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
19. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
20. ? @ 0x00007fe94d0adac3
21. ? @ 0x00007fe94d13f850

2024.11.04 23:22:25.143937 [ 2935 ] {2614e897-ca36-4151-a814-69c6119477a4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:25.165268 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:25.756446 [ 7099 ] {544434ea-85d7-4b40-9f1e-53dea177a4db} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49726) (comment: 02337_analyzer_columns_basic.sql) (in query: SELECT test_table.id FROM test_table;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:25.762945 [ 7099 ] {544434ea-85d7-4b40-9f1e-53dea177a4db} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:26.098644 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:26.282050 [ 7099 ] {020dca82-c8ea-4d82-b73b-ca17362b1c9a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49750) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:26.286167 [ 7099 ] {020dca82-c8ea-4d82-b73b-ca17362b1c9a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:27.300314 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:27.306550 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:27.449406 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_128_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=4): (while reading column initial_query_start_time): (while reading from part /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/202411_1_122_21/ in table system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:27.574446 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:29.184669 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_128_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=30). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:30.481697 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_128_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:30.607860 [ 2935 ] {fc8d82bd-eff0-424f-8358-17724413fc6d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49794) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:30.612044 [ 2935 ] {fc8d82bd-eff0-424f-8358-17724413fc6d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:31.234176 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:31.391836 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_128_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:31.815834 [ 2935 ] {0650ea02-2235-4fd2-a4d5-36b680047a1a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49796) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:31.820425 [ 2935 ] {0650ea02-2235-4fd2-a4d5-36b680047a1a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:33.882950 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:34.195238 [ 3692 ] {} <Warning> CgroupsMemoryUsageObserver: Exceeded hard memory limit (14.54 GiB)
2024.11.04 23:22:35.012544 [ 2935 ] {42130014-a339-4d78-85c3-484adfcb3c71} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44154) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:35.015892 [ 2935 ] {42130014-a339-4d78-85c3-484adfcb3c71} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=27, jobs=27). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:36.271843 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_129_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:176: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b005
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:36.310618 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/uta/vslwcmhlexyazdqtrwouonhwpxsfb, total size 385875968, count 385875968, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 4f43e3dc-581d-4a49-8f10-8cee48c8658b, upload has finished false.
2024.11.04 23:22:36.310871 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/uta/vslwcmhlexyazdqtrwouonhwpxsfb, total size 385875968, count 385875968, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 4f43e3dc-581d-4a49-8f10-8cee48c8658b, upload has finished false
2024.11.04 23:22:36.407658 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/uta/vslwcmhlexyazdqtrwouonhwpxsfb, total size 385875968, count 385875968, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 4f43e3dc-581d-4a49-8f10-8cee48c8658b, upload has finished false
2024.11.04 23:22:36.414018 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 58 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:37.311716 [ 2935 ] {b79bc2b1-c948-41e5-928d-da9b0f5c1b98} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44162) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:37.315616 [ 2935 ] {b79bc2b1-c948-41e5-928d-da9b0f5c1b98} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:37.535875 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_129_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:39.098609 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:39.501820 [ 2935 ] {e4cb947c-2f53-4399-a1db-8e7f2523bbfe} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44192) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:39.507027 [ 2935 ] {e4cb947c-2f53-4399-a1db-8e7f2523bbfe} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:22:41.810471 [ 2935 ] {aa06f883-14e7-4aeb-8d90-011e250ba000} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44224) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:41.814735 [ 2935 ] {aa06f883-14e7-4aeb-8d90-011e250ba000} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:42.281781 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_129_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:42.874852 [ 2935 ] {5c1871fe-a30c-4be6-b218-3fb240ba07f8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44230) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:42.882076 [ 2935 ] {5c1871fe-a30c-4be6-b218-3fb240ba07f8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:43.777234 [ 3738 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:22:43.777355 [ 3738 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:22:44.466577 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/qcr/wioyvqqegioxahwhglilecjvacome, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6716905a-a4cc-4f83-bb68-5e175690ad0e, upload has finished false.
2024.11.04 23:22:44.466849 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/qcr/wioyvqqegioxahwhglilecjvacome, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6716905a-a4cc-4f83-bb68-5e175690ad0e, upload has finished false
2024.11.04 23:22:44.661537 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/qcr/wioyvqqegioxahwhglilecjvacome, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 6716905a-a4cc-4f83-bb68-5e175690ad0e, upload has finished false
2024.11.04 23:22:44.665450 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 50 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:46.197507 [ 2935 ] {d86a0b60-7235-4b68-81c0-f94a9ce6a4de} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:42936) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:46.201286 [ 2935 ] {d86a0b60-7235-4b68-81c0-f94a9ce6a4de} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:46.708055 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 67 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:46.839807 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:47.245658 [ 2935 ] {6595a3b5-1e22-40b0-a2f1-be47e67cdf15} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:42938) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:47.249702 [ 2935 ] {6595a3b5-1e22-40b0-a2f1-be47e67cdf15} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:48.089887 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:49.465386 [ 2935 ] {39caac7c-5e57-460c-a3df-f22576ab7498} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:42964) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:49.469048 [ 2935 ] {39caac7c-5e57-460c-a3df-f22576ab7498} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:50.668926 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:51.788152 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:52.783983 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:53.443156 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:53.713192 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ype/dsmxtcrnldgxvqxnelpxenmksldxy, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f2c57695-d5cb-47df-8346-ef374cfcfb3c, upload has finished false.
2024.11.04 23:22:53.713414 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ype/dsmxtcrnldgxvqxnelpxenmksldxy, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f2c57695-d5cb-47df-8346-ef374cfcfb3c, upload has finished false
2024.11.04 23:22:53.737916 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ype/dsmxtcrnldgxvqxnelpxenmksldxy, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f2c57695-d5cb-47df-8346-ef374cfcfb3c, upload has finished false
2024.11.04 23:22:53.743383 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 80 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:54.433022 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:54.988458 [ 2935 ] {4c6cd0dc-3a37-4a59-8b3f-4205b2f1b9e3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57680) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:54.992127 [ 2935 ] {4c6cd0dc-3a37-4a59-8b3f-4205b2f1b9e3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:55.289986 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:55.292826 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:55.851250 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:56.082022 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:57.696053 [ 2935 ] {62d36e24-b10c-4222-8aa1-30fb047e6e31} <Error> executeQuery: Code: 47. DB::Exception: Identifier 't1.d' cannot be resolved from table with name test_13.t1. In scope SELECT * FROM t1 INNER JOIN s ON t1.k = s.k INNER JOIN t2 ON t2.x = t1.x WHERE t1.d >= now(). Maybe you meant: ['t1.k']. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57700) (comment: 02694_wrong_identifier_shouldnt_be_accepted.sql) (in query: SELECT * FROM t1 INNER JOIN s ON t1.k = s.k INNER JOIN t2 ON t2.x = t1.x WHERE (t1.d >= now());), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, String&, String, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String const&, String&, String&&, String&&) @ 0x00000000348f7dff
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:3216: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromStorage(DB::Identifier const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::TableExpressionData const&, DB::(anonymous namespace)::IdentifierResolveScope&, unsigned long, bool) @ 0x000000003486220b
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTreeNode(DB::(anonymous namespace)::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000348483b5
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:3436: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTreeNode(DB::(anonymous namespace)::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000348470e6
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:3436: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTreeNode(DB::(anonymous namespace)::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000348470e6
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTree(DB::(anonymous namespace)::IdentifierLookup const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347f12c2
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifier(DB::(anonymous namespace)::IdentifierLookup const&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::IdentifierResolveSettings) @ 0x00000000347eaba1
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:254: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347bb970
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003480890d
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034799761
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:22:59.659091 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:59.659260 [ 2935 ] {62d36e24-b10c-4222-8aa1-30fb047e6e31} <Error> TCPHandler: Code: 47. DB::Exception: Identifier 't1.d' cannot be resolved from table with name test_13.t1. In scope SELECT * FROM t1 INNER JOIN s ON t1.k = s.k INNER JOIN t2 ON t2.x = t1.x WHERE t1.d >= now(). Maybe you meant: ['t1.k']. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, String&, String, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String const&, String&, String&&, String&&) @ 0x00000000348f7dff
3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:3216: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromStorage(DB::Identifier const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::TableExpressionData const&, DB::(anonymous namespace)::IdentifierResolveScope&, unsigned long, bool) @ 0x000000003486220b
4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTreeNode(DB::(anonymous namespace)::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000348483b5
5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:3436: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTreeNode(DB::(anonymous namespace)::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000348470e6
6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:3436: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTreeNode(DB::(anonymous namespace)::IdentifierLookup const&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000348470e6
7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifierFromJoinTree(DB::(anonymous namespace)::IdentifierLookup const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347f12c2
8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::tryResolveIdentifier(DB::(anonymous namespace)::IdentifierLookup const&, DB::(anonymous namespace)::IdentifierResolveScope&, DB::(anonymous namespace)::IdentifierResolveSettings) @ 0x00000000347eaba1
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:254: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347bb970
10. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5548: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003480890d
12. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034799761
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:22:59.659295 [ 3948 ] {d45548e0-1357-4232-baa8-dedb9739d66c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57722) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:22:59.663016 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:22:59.663039 [ 3948 ] {d45548e0-1357-4232-baa8-dedb9739d66c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:00.392630 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/dhy/tlenczkeuxzeqwwtnfxjswhygogzv, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fa3bfea2-ca5a-4e65-bf21-62ad12d53750, upload has finished false.
2024.11.04 23:23:00.392888 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/dhy/tlenczkeuxzeqwwtnfxjswhygogzv, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fa3bfea2-ca5a-4e65-bf21-62ad12d53750, upload has finished false
2024.11.04 23:23:00.419115 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/dhy/tlenczkeuxzeqwwtnfxjswhygogzv, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id fa3bfea2-ca5a-4e65-bf21-62ad12d53750, upload has finished false
2024.11.04 23:23:00.497969 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 41 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:00.654158 [ 2935 ] {8934e5aa-a48b-4d22-ae3b-6f61056984af} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57734) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:00.658144 [ 2935 ] {8934e5aa-a48b-4d22-ae3b-6f61056984af} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=19, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:01.750706 [ 3948 ] {f26c1d6b-1d6a-490e-b040-86d6f09c0821} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57764) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:01.754879 [ 3948 ] {f26c1d6b-1d6a-490e-b040-86d6f09c0821} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:02.883007 [ 2935 ] {bfa961f4-67ad-487c-b056-4325f6317dd9} <Error> executeQuery: Code: 565. DB::Exception: Too many partitions to read. Current 2, max 1. (TOO_MANY_PARTITIONS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57760) (comment: 01632_max_partitions_to_read.sql) (in query: select * from p order by i;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<unsigned long, DB::SettingFieldNumber<long>&>(int, FormatStringHelperImpl<std::type_identity<unsigned long>::type, std::type_identity<DB::SettingFieldNumber<long>&>::type>, unsigned long&&, DB::SettingFieldNumber<long>&) @ 0x000000003946a9ac
3. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:852: DB::MergeTreeDataSelectExecutor::checkLimits(DB::MergeTreeData const&, DB::ReadFromMergeTree::AnalysisResult const&, std::shared_ptr<DB::Context const> const&) @ 0x000000003945757e
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6f0c39
5. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
6. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:23:02.883075 [ 3948 ] {9e256706-b74e-4d50-98f5-729d25aa4633} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57778) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:02.886727 [ 3948 ] {9e256706-b74e-4d50-98f5-729d25aa4633} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:02.960180 [ 2935 ] {bfa961f4-67ad-487c-b056-4325f6317dd9} <Error> TCPHandler: Code: 565. DB::Exception: Too many partitions to read. Current 2, max 1. (TOO_MANY_PARTITIONS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<unsigned long, DB::SettingFieldNumber<long>&>(int, FormatStringHelperImpl<std::type_identity<unsigned long>::type, std::type_identity<DB::SettingFieldNumber<long>&>::type>, unsigned long&&, DB::SettingFieldNumber<long>&) @ 0x000000003946a9ac
3. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:852: DB::MergeTreeDataSelectExecutor::checkLimits(DB::MergeTreeData const&, DB::ReadFromMergeTree::AnalysisResult const&, std::shared_ptr<DB::Context const> const&) @ 0x000000003945757e
4. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6f0c39
5. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
6. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:23:03.257751 [ 2935 ] {dbb7d0c2-42e0-441f-8b88-c2e310575548} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:57760) (comment: 01632_max_partitions_to_read.sql) (in query: select * from p order by i settings max_partitions_to_read = 0;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:23:03.259280 [ 2935 ] {dbb7d0c2-42e0-441f-8b88-c2e310575548} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:0: DB::MergeTreeDataSelectExecutor::filterPartsByPrimaryKeyAndSkipIndexes(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>&&, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>&&, std::shared_ptr<DB::StorageInMemoryMetadata const>, std::shared_ptr<DB::Context const> const&, DB::KeyCondition const&, std::optional<DB::KeyCondition> const&, DB::UsefulSkipIndexes const&, DB::MergeTreeReaderSettings const&, std::shared_ptr<Poco::Logger>, unsigned long, std::vector<DB::ReadFromMergeTree::IndexStat, std::allocator<DB::ReadFromMergeTree::IndexStat>>&, bool) @ 0x000000003944d953
7. ./contrib/llvm-project/libcxx/include/vector:951: DB::ReadFromMergeTree::selectRangesToReadImpl(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6dcb14
8. ./contrib/llvm-project/libcxx/include/vector:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo const&, std::shared_ptr<DB::Context const>, unsigned long, std::shared_ptr<std::unordered_map<String, long, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, long>>>>, DB::MergeTreeData const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<Poco::Logger>, std::optional<DB::ReadFromMergeTree::Indexes>&) @ 0x000000003b6d1172
9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(std::vector<std::shared_ptr<DB::IMergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::IMergeTreeDataPart const>>>, std::vector<std::shared_ptr<DB::AlterConversions const>, std::allocator<std::shared_ptr<DB::AlterConversions const>>>) const @ 0x000000003b6d0496
10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1854: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000003b6e8088
11. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b6ee66a
12. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
13. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x0000000035736d2d
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x0000000035736741
16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
17. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
19. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
23. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
24. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
25. ? @ 0x00007fe94d0adac3
26. ? @ 0x00007fe94d13f850

2024.11.04 23:23:04.033128 [ 2935 ] {950047a4-841d-4bd6-bf48-b234cc79199b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46296) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:04.036054 [ 2935 ] {950047a4-841d-4bd6-bf48-b234cc79199b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:04.503731 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:23:04.503884 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:23:05.550955 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/zun/txyyvptkjukbyqsqblwdfxzjztjiu, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id aa9e745b-d98d-49ab-9f84-11538c4559a3, upload has finished false.
2024.11.04 23:23:05.551148 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/zun/txyyvptkjukbyqsqblwdfxzjztjiu, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id aa9e745b-d98d-49ab-9f84-11538c4559a3, upload has finished false
2024.11.04 23:23:05.560887 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/zun/txyyvptkjukbyqsqblwdfxzjztjiu, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id aa9e745b-d98d-49ab-9f84-11538c4559a3, upload has finished false
2024.11.04 23:23:05.569309 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 10 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:05.632831 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:06.374024 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:06.900944 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:07.413621 [ 2935 ] {7a3e73fd-4bde-4517-a37b-80c71eb6a4af} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46358) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:07.426691 [ 2935 ] {7a3e73fd-4bde-4517-a37b-80c71eb6a4af} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:07.939499 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 25 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:08.082391 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:08.488067 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:09.434782 [ 2935 ] {c5485d00-b060-499b-9b9a-53743311b929} <Error> executeQuery: Code: 47. DB::Exception: Missing columns: 'String' while processing query: 'String AS b', required columns: 'String' 'String': default expression and column type are incompatible. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46368) (comment: 02287_ephemeral_format_crash.sql) (in query: CREATE TABLE test(a UInt8, b EPHEMERAL String) Engine=Memory();), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/ColumnsDescription.cpp:947: DB::validateColumnsDefaultsAndGetSampleBlock(std::shared_ptr<DB::IAST>, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const>) @ 0x00000000379f265c
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x000000003461b646
7. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:23:09.436093 [ 2935 ] {c5485d00-b060-499b-9b9a-53743311b929} <Error> TCPHandler: Code: 47. DB::Exception: Missing columns: 'String' while processing query: 'String AS b', required columns: 'String' 'String': default expression and column type are incompatible. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/ColumnsDescription.cpp:947: DB::validateColumnsDefaultsAndGetSampleBlock(std::shared_ptr<DB::IAST>, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const>) @ 0x00000000379f265c
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x000000003461b646
7. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:23:09.541782 [ 2935 ] {095a02e3-7d60-4eb6-bdac-6f0075ceadd8} <Error> executeQuery: Code: 47. DB::Exception: Missing columns: 'test' while processing query: '_CAST(test AS b_tmp_alter11266262340712284841, 'String') AS b, b_tmp_alter11266262340712284841', required columns: 'test' 'test': default expression and column type are incompatible. (UNKNOWN_IDENTIFIER) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46368) (comment: 02287_ephemeral_format_crash.sql) (in query: CREATE TABLE test(a UInt8, b String EPHEMERAL test) Engine=Memory();), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/ColumnsDescription.cpp:947: DB::validateColumnsDefaultsAndGetSampleBlock(std::shared_ptr<DB::IAST>, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const>) @ 0x00000000379f265c
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x000000003461b646
7. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:23:09.542996 [ 2935 ] {095a02e3-7d60-4eb6-bdac-6f0075ceadd8} <Error> TCPHandler: Code: 47. DB::Exception: Missing columns: 'test' while processing query: '_CAST(test AS b_tmp_alter11266262340712284841, 'String') AS b, b_tmp_alter11266262340712284841', required columns: 'test' 'test': default expression and column type are incompatible. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008ec447b
3. ./build_docker/./src/Interpreters/TreeRewriter.cpp:1252: DB::TreeRewriterResult::collectUsedColumns(std::shared_ptr<DB::IAST> const&, bool, bool) @ 0x00000000362e64c7
4. ./build_docker/./src/Interpreters/TreeRewriter.cpp:0: DB::TreeRewriter::analyze(std::shared_ptr<DB::IAST>&, DB::NamesAndTypesList const&, std::shared_ptr<DB::IStorage const>, std::shared_ptr<DB::StorageSnapshot> const&, bool, bool, bool, bool) const @ 0x000000003630aad2
5. ./build_docker/./src/Storages/ColumnsDescription.cpp:947: DB::validateColumnsDefaultsAndGetSampleBlock(std::shared_ptr<DB::IAST>, DB::NamesAndTypesList const&, std::shared_ptr<DB::Context const>) @ 0x00000000379f265c
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::shared_ptr<DB::Context const>, bool, bool) @ 0x000000003461b646
7. ./contrib/boost/boost/multi_index_container.hpp:1000: DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x00000000346228f8
8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1224: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000003463ec9b
9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:23:09.674835 [ 3948 ] {643a5cbd-5296-4f28-bc52-9c74c6f729ef} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46388) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:09.678611 [ 3948 ] {643a5cbd-5296-4f28-bc52-9c74c6f729ef} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:10.729366 [ 2935 ] {b47b5b9c-7fad-4ca7-9521-8ec3ca055bbc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46402) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:10.807955 [ 2935 ] {b47b5b9c-7fad-4ca7-9521-8ec3ca055bbc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:13.186292 [ 2935 ] {bb5e29bf-dcdc-4f82-8b62-28d6179703b0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:46428) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:13.190138 [ 2935 ] {bb5e29bf-dcdc-4f82-8b62-28d6179703b0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:13.902560 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:14.643567 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/glt/ublxxqgmedavemxthomjwvmtatngq, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 007de20c-d351-48fc-b408-89dc8f12c0f9, upload has finished false.
2024.11.04 23:23:14.643876 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/glt/ublxxqgmedavemxthomjwvmtatngq, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 007de20c-d351-48fc-b408-89dc8f12c0f9, upload has finished false
2024.11.04 23:23:14.678997 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/glt/ublxxqgmedavemxthomjwvmtatngq, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 007de20c-d351-48fc-b408-89dc8f12c0f9, upload has finished false
2024.11.04 23:23:14.683274 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 2 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:15.752256 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 13 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:16.571572 [ 3948 ] {6f9c338d-e0e2-41e2-bace-3b41f16f5aa0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58612) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:16.575046 [ 3948 ] {6f9c338d-e0e2-41e2-bace-3b41f16f5aa0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:17.994828 [ 2935 ] {82124583-c669-4d24-9768-a2a56ccb333e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58568) (comment: 02016_bit_shift_right_for_string_integer.sql) (in query: SELECT 0, toFixedString('Hello', 10), bin(bitShiftRight(toFixedString('Hello', 10), 0));), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:23:17.997544 [ 2935 ] {82124583-c669-4d24-9768-a2a56ccb333e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:23:18.754254 [ 12459 ] {2211b10e-85f7-46f3-8c98-a9f71e75ffa0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58642) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:18.758340 [ 12459 ] {2211b10e-85f7-46f3-8c98-a9f71e75ffa0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:19.435239 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:20.489690 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:21.616348 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:22.133242 [ 3948 ] {d23ff4cc-c6e2-47c7-aab0-22e15995e901} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58682) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:22.136439 [ 3948 ] {d23ff4cc-c6e2-47c7-aab0-22e15995e901} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:22.932280 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:23.188166 [ 3948 ] {70804085-7e37-41a2-9c9f-fe0d71166ffd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58706) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:23.191263 [ 3948 ] {70804085-7e37-41a2-9c9f-fe0d71166ffd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:23.468179 [ 3948 ] {1721345a-1362-4a28-94a2-f5e39d4b444d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58714) (comment: 01945_system_warnings.sh) (in query: SELECT count() FROM system.warnings WHERE message LIKE '%Obsolete setting%'), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:23.475250 [ 3948 ] {1721345a-1362-4a28-94a2-f5e39d4b444d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:23.766340 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:24.855658 [ 3948 ] {487c2cb0-9968-4494-8fa1-826511a109db} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36838) (comment: 02493_max_streams_for_merge_tree_reading.sql) (in query: insert into t select number from numbers_mt(10000000) settings max_insert_threads=8;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:23:25.028436 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/hbi/xhtcrnfyicxjdcecclxxdvjzmiurg, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id feb07a12-b94c-4772-ab90-7553e3d3ad87, upload has finished false.
2024.11.04 23:23:25.028661 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/hbi/xhtcrnfyicxjdcecclxxdvjzmiurg, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id feb07a12-b94c-4772-ab90-7553e3d3ad87, upload has finished false
2024.11.04 23:23:25.072481 [ 3948 ] {487c2cb0-9968-4494-8fa1-826511a109db} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850

2024.11.04 23:23:25.171230 [ 3696 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/hbi/xhtcrnfyicxjdcecclxxdvjzmiurg, total size 285212672, count 285212672, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id feb07a12-b94c-4772-ab90-7553e3d3ad87, upload has finished false
2024.11.04 23:23:25.176886 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 28 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:25.341313 [ 3948 ] {bbd75d5a-1a97-4849-8817-5b7e8e7cf609} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36854) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:25.345418 [ 3948 ] {bbd75d5a-1a97-4849-8817-5b7e8e7cf609} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=12, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:25.742087 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:26.474917 [ 12459 ] {1de70130-f9f1-4ab5-9f2b-6b892f5cca8c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36868) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:26.479084 [ 12459 ] {1de70130-f9f1-4ab5-9f2b-6b892f5cca8c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:26.833179 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:27.100287 [ 3948 ] {844b4f4e-ad4f-4697-8e6d-a5756de74c65} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36858) (comment: 02324_map_combinator_bug.sql) (in query: SELECT ignore(CAST((arrayMap(k -> toString(k), mapKeys(uniqMapMerge(uniq_tags_ids) AS m)), mapValues(m)), 'Map(String, UInt32)')) FROM segfault_mv;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:27.110498 [ 3948 ] {844b4f4e-ad4f-4697-8e6d-a5756de74c65} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:27.500844 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:27.563712 [ 3948 ] {fc1bc737-ef79-4ef1-85a0-100d7db9173b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36882) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:27.598348 [ 3948 ] {fc1bc737-ef79-4ef1-85a0-100d7db9173b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:29.761884 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/sgt/kcrnxtpykyhqxxtcyhrnbwkresznt, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ec1ad1f6-7b2e-415c-86ce-837dacfbad1d, upload has finished false.
2024.11.04 23:23:29.762037 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/sgt/kcrnxtpykyhqxxtcyhrnbwkresznt, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ec1ad1f6-7b2e-415c-86ce-837dacfbad1d, upload has finished false
2024.11.04 23:23:29.767108 [ 3698 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/sgt/kcrnxtpykyhqxxtcyhrnbwkresznt, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ec1ad1f6-7b2e-415c-86ce-837dacfbad1d, upload has finished false
2024.11.04 23:23:29.775703 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. ./src/IO/WriteBuffer.h:65: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039408090
17. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&) @ 0x00000000393f910f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f5c43
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:30.161811 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_126_27}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:30.298459 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:30.708440 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:30.732809 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_126_27}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:31.052904 [ 3948 ] {411b1809-e2e9-4770-8ee3-ad8599b100fa} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36924) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:31.056673 [ 3948 ] {411b1809-e2e9-4770-8ee3-ad8599b100fa} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:32.437424 [ 3948 ] {93094f63-1032-439a-bd10-b32945c4ed7e} <Error> executeQuery: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:36938) (in query: select _path, _file from file('nonexist.txt', 'CSV', 'val1 char')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:23:32.438854 [ 3948 ] {93094f63-1032-439a-bd10-b32945c4ed7e} <Error> TCPHandler: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:23:33.886643 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=27). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:34.244111 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:34.342487 [ 12459 ] {08e20042-5153-4d33-91b3-391c7d1454f1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38668) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:34.345523 [ 12459 ] {08e20042-5153-4d33-91b3-391c7d1454f1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:35.374619 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:35.441381 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/izl/erzdaxupzncxxfrtskzhlgvnpycop, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 567dc5dd-439e-42e2-934a-b75ccc92b6af, upload has finished false.
2024.11.04 23:23:35.441639 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/izl/erzdaxupzncxxfrtskzhlgvnpycop, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 567dc5dd-439e-42e2-934a-b75ccc92b6af, upload has finished false
2024.11.04 23:23:35.457381 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/izl/erzdaxupzncxxfrtskzhlgvnpycop, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 567dc5dd-439e-42e2-934a-b75ccc92b6af, upload has finished false
2024.11.04 23:23:35.460361 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::writePart(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030fa30bc
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::nextImpl() @ 0x0000000030f882b3
11. ./src/IO/WriteBuffer.h:65: DB::WriteBufferFromFileDecorator::nextImpl() @ 0x0000000032473303
12. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
13. DB::WriteBuffer::write(char const*, unsigned long) @ 0x0000000007ca2cad
14. ./build_docker/./src/Compression/CompressedWriteBuffer.cpp:56: DB::CompressedWriteBuffer::nextImpl() @ 0x00000000316eecfe
15. ./src/IO/WriteBuffer.h:65: DB::HashingWriteBuffer::nextImpl() @ 0x0000000038ddd822
16. ./src/IO/WriteBuffer.h:65: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039408090
17. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::getCurrentMarksForColumn(DB::NameAndTypePair const&, std::set<String, std::less<String>, std::allocator<String>>&) @ 0x00000000393f910f
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::writeColumn(DB::NameAndTypePair const&, DB::IColumn const&, std::set<String, std::less<String>, std::allocator<String>>&, std::vector<DB::Granule, std::allocator<DB::Granule>> const&) @ 0x00000000393f5c43
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:291: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const*) @ 0x00000000393f08fb
22. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:338: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x00000000398853a9
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b76
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:36.287755 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:36.762312 [ 3948 ] {73e766ad-919e-4d1e-84f3-4e69296f0fb9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1011). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38712) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:23:36.852568 [ 3948 ] {73e766ad-919e-4d1e-84f3-4e69296f0fb9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1011). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:23:38.304947 [ 3948 ] {31e18f00-0c43-4f6e-bb1b-ffb0ef757c08} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38716) (comment: 02661_read_from_archive_tarbzip2.sh) (in query: SELECT * FROM file('/var/lib/clickhouse/user_files/02661_read_from_archive_test_13_tarbz2_archive{1..2}.tar.bz2 :: 02661_read_from_archive_test_13_tarbz2_data1.csv') ORDER BY 1, 2), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:38.304979 [ 12459 ] {30d129ce-47ec-43ea-a1e5-e046c2b16bc1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38724) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:38.309274 [ 12459 ] {30d129ce-47ec-43ea-a1e5-e046c2b16bc1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:38.319587 [ 3948 ] {31e18f00-0c43-4f6e-bb1b-ffb0ef757c08} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:38.400813 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 23 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:38.810151 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column partitions): (while reading from part /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/202411_1_122_21/ in table system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:39.253493 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:41.374964 [ 3948 ] {0c7ed023-040c-4030-8625-c96e77ac500d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38776) (comment: 02661_read_from_archive_tarbzip2.sh) (in query: SELECT * FROM 02661_archive_table ORDER BY 1, 2), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:41.378537 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=31). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:41.384758 [ 3948 ] {0c7ed023-040c-4030-8625-c96e77ac500d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:41.386703 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:41.740123 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:41.972851 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 23 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:42.592152 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:42.783336 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:45.016459 [ 3948 ] {dc59a099-8374-453d-a36a-91524b884a8c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37850) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:45.020154 [ 3948 ] {dc59a099-8374-453d-a36a-91524b884a8c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:45.131272 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:45.339622 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 86 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:45.623438 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=9): (while reading column initial_query_start_time_microseconds): (while reading from part /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/202411_1_122_21/ in table system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:46.659919 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:47.226600 [ 3948 ] {b954c6ca-92d8-49e2-8633-44ba527089a4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37878) (comment: 02661_read_from_archive_tarbzip2.sh) (in query: SELECT * FROM file('/var/lib/clickhouse/user_files/02661_read_from_archive_test_13_tarbz2_archive3.tar.bz2 :: 02661_read_from_archive_test_13_tarbz2_data*.csv') ORDER BY 1, 2), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:47.235484 [ 3948 ] {b954c6ca-92d8-49e2-8633-44ba527089a4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:47.248161 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {0121b8e0-50ee-471c-82d2-e88f58eee0fd::202411_1_130_26}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:47.342792 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 9 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:48.862273 [ 3948 ] {4fe39d8b-4cee-4a70-8bed-e8fa393d4653} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37908) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:48.865115 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:48.866217 [ 3948 ] {4fe39d8b-4cee-4a70-8bed-e8fa393d4653} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:49.280874 [ 3948 ] {78b26541-432a-47de-bcf7-03d97484c004} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37932) (comment: 02661_read_from_archive_tarbzip2.sh) (in query: SELECT * FROM 02661_archive_table ORDER BY 1, 2), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:49.285635 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:49.290866 [ 3948 ] {78b26541-432a-47de-bcf7-03d97484c004} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=30, jobs=30). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:51.089146 [ 3948 ] {bb156de9-fcdf-4910-a259-83e085ed4d25} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37938) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:51.091597 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=36). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:51.092604 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:51.093330 [ 3948 ] {bb156de9-fcdf-4910-a259-83e085ed4d25} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:52.509321 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:52.509348 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:52.509622 [ 12459 ] {30791c80-ae28-4c0c-bec3-8e197f804da5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37974) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:52.513604 [ 12459 ] {30791c80-ae28-4c0c-bec3-8e197f804da5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:52.514562 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:52.515795 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:52.766378 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:23:52.766500 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:23:52.811390 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=28). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:53.567283 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/lka/idvizvkrrtlgwhmekohztvkewburb, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 05593e6e-56cd-437e-849f-e7ccc8aaab8d, upload has finished false.
2024.11.04 23:23:53.567443 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/lka/idvizvkrrtlgwhmekohztvkewburb, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 05593e6e-56cd-437e-849f-e7ccc8aaab8d, upload has finished false
2024.11.04 23:23:53.598812 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=46). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:53.607797 [ 3707 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/lka/idvizvkrrtlgwhmekohztvkewburb, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 05593e6e-56cd-437e-849f-e7ccc8aaab8d, upload has finished false
2024.11.04 23:23:53.614389 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 59 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:53.646630 [ 12459 ] {421cd5a9-4f4e-47a2-98f4-cf8b5f1cc70c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53706) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:53.649883 [ 12459 ] {421cd5a9-4f4e-47a2-98f4-cf8b5f1cc70c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:54.427866 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:55.592360 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=8): (while reading column query_cache_usage): (while reading from part /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/202411_1_122_21/ in table system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:57.178754 [ 12459 ] {57ab20d9-261c-4e61-a505-0935d7b22607} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53794) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:57.183243 [ 12459 ] {57ab20d9-261c-4e61-a505-0935d7b22607} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:57.184184 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=38). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c65901
13. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:641: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cfe010
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
15. ./build_docker/./src/DataTypes/Serializations/SerializationMap.cpp:0: DB::SerializationMap::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c63e21
16. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
17. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
18. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
19. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
22. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
23. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
24. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
29. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:57.214008 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 4 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:57.477369 [ 7099 ] {2bd6fa6b-66aa-4cef-ab56-401d6231d535} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53796) (comment: 02661_read_from_archive_tarbzip2.sh) (in query: SELECT * FROM 02661_archive_table ORDER BY 1, 2), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:57.488972 [ 7099 ] {2bd6fa6b-66aa-4cef-ab56-401d6231d535} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:23:57.701967 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:58.601895 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=105). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c65901
14. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:641: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cfe010
15. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
17. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
18. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
21. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
22. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:58.616758 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_9_9_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 1 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:58.863491 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:23:59.385977 [ 12459 ] {0426776f-4810-4b23-89a5-ed0be48b370b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53826) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:59.396528 [ 12459 ] {0426776f-4810-4b23-89a5-ed0be48b370b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:23:59.569290 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=36). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:00.266494 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_135_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=78). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:00.514418 [ 12459 ] {61203b82-d323-441c-a4b3-d6537bc1e9dc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53836) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:00.518824 [ 12459 ] {61203b82-d323-441c-a4b3-d6537bc1e9dc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:01.569475 [ 12459 ] {d26aebcd-e2e0-4504-8aa7-8af0a7ab24cd} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53840) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:01.573511 [ 12459 ] {d26aebcd-e2e0-4504-8aa7-8af0a7ab24cd} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:02.491559 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:03.569255 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:24:03.569431 [ 3732 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:24:03.950645 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/hiq/ztztlyqhmnysvvpbmygiuupsvavqr, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 18f9a033-6e79-4ec5-bc56-3a81d693bd86, upload has finished false.
2024.11.04 23:24:03.950966 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/hiq/ztztlyqhmnysvvpbmygiuupsvavqr, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 18f9a033-6e79-4ec5-bc56-3a81d693bd86, upload has finished false
2024.11.04 23:24:03.962177 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/hiq/ztztlyqhmnysvvpbmygiuupsvavqr, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 18f9a033-6e79-4ec5-bc56-3a81d693bd86, upload has finished false
2024.11.04 23:24:04.228281 [ 12459 ] {2292e2db-22e2-4c23-90cd-b348bc006c8b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53850) (comment: 01603_remove_column_ttl.sql) (in query: INSERT INTO table_with_column_ttl VALUES ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:24:04.229080 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 18 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:04.231573 [ 12459 ] {2292e2db-22e2-4c23-90cd-b348bc006c8b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processInsertQuery() @ 0x000000003a4f6b50
28. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6f91
29. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981

2024.11.04 23:24:04.238718 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:04.756465 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:06.937142 [ 7099 ] {b5acd440-dd5e-4305-8f5e-c6c5421bdf15} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38316) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:06.937204 [ 12459 ] {d293eaf9-564e-410c-83e4-dde44465ddfb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38302) (comment: 01825_type_json_17.sql) (in query: SELECT obj FROM t_json_17 ORDER BY obj.id FORMAT JSONEachRow;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c65901
13. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:641: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cfe010
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
16. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
17. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
25. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
26. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
28. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f

2024.11.04 23:24:06.950073 [ 12459 ] {d293eaf9-564e-410c-83e4-dde44465ddfb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c65901
13. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:641: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cfe010
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
16. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
17. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr<DB::MergeTreeReadTaskInfo const> const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000003978f0ed
20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolBase.cpp:0: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr<DB::MergeTreeReadTaskInfo const>, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000003b77fe0f
21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:70: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000003b81a829
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000003b78c1f3
23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:122: DB::MergeTreeSelectProcessor::read() @ 0x000000003974e117
24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:181: DB::MergeTreeSource::tryGenerate() @ 0x000000003b738d60
25. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
26. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
28. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f

2024.11.04 23:24:06.966002 [ 7099 ] {b5acd440-dd5e-4305-8f5e-c6c5421bdf15} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:07.054141 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {54259a93-f7ca-4ac1-8b7b-0c01fa06532c::202411_1_146_29}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:07.639782 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=28). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:07.929308 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:08.065101 [ 12459 ] {a9bcb01b-6975-4b3b-b813-5f4e6fd97f06} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38340) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:08.068731 [ 12459 ] {a9bcb01b-6975-4b3b-b813-5f4e6fd97f06} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:08.850998 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:09.189565 [ 7099 ] {00adae59-6275-48b3-b698-d1295584d0f4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38346) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:09.266301 [ 7099 ] {00adae59-6275-48b3-b698-d1295584d0f4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:09.796058 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:10.172309 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/hex/rrwpgmkkzonwilvndyayurdocxudf, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2fd4c776-503a-4acf-879e-a46a9af04903, upload has finished false.
2024.11.04 23:24:10.172525 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/hex/rrwpgmkkzonwilvndyayurdocxudf, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2fd4c776-503a-4acf-879e-a46a9af04903, upload has finished false
2024.11.04 23:24:10.214434 [ 3709 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/hex/rrwpgmkkzonwilvndyayurdocxudf, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2fd4c776-503a-4acf-879e-a46a9af04903, upload has finished false
2024.11.04 23:24:10.238386 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 14 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:11.615649 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:11.615729 [ 12459 ] {e85aaedd-0c03-44c5-8983-e80b379e77d2} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toMonday is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toMonday(toDate(16433), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toMonday(toDate(16433), 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToMondayImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b73dd54
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToMondayImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b73656e
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:11.617297 [ 12459 ] {e85aaedd-0c03-44c5-8983-e80b379e77d2} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toMonday is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toMonday(toDate(16433), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToMondayImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b73dd54
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToMondayImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b73656e
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:13.845397 [ 12459 ] {d023d106-5d89-4e11-a708-a7676209a97e} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toStartOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toStartOfWeek(toDate(16433), 0, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toStartOfWeek(toDate(16433), 0, 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToStartOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b68fc78
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToStartOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b687057
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:13.847698 [ 12459 ] {d023d106-5d89-4e11-a708-a7676209a97e} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toStartOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toStartOfWeek(toDate(16433), 0, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToStartOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b68fc78
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToStartOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b687057
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:13.877312 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 92 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:14.327382 [ 12459 ] {ddf7ab4a-a64f-453a-88bb-9c39eb04d147} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toStartOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toStartOfWeek(toDate(16433), 1, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toStartOfWeek(toDate(16433), 1, 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToStartOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b68fc78
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToStartOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b687057
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:14.394219 [ 12459 ] {ddf7ab4a-a64f-453a-88bb-9c39eb04d147} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toStartOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toStartOfWeek(toDate(16433), 1, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToStartOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b68fc78
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToStartOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b687057
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:16.182710 [ 12459 ] {2f110375-8610-4c87-aac8-c863ba1f84f8} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toLastDayOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toLastDayOfWeek(toDate(16433), 0, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toLastDayOfWeek(toDate(16433), 0, 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToLastDayOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b69adf8
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToLastDayOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b692a57
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:16.182746 [ 7099 ] {641653e3-90c4-4c30-b7b9-c57009b861f0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54130) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:16.182772 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:16.184602 [ 12459 ] {2f110375-8610-4c87-aac8-c863ba1f84f8} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toLastDayOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toLastDayOfWeek(toDate(16433), 0, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToLastDayOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b69adf8
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToLastDayOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b692a57
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:16.186381 [ 7099 ] {641653e3-90c4-4c30-b7b9-c57009b861f0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:16.201444 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:16.994888 [ 12459 ] {2cecf4e7-2064-4656-8035-0a060c27a9f9} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toLastDayOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toLastDayOfWeek(toDate(16433), 1, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toLastDayOfWeek(toDate(16433), 1, 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToLastDayOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b69adf8
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToLastDayOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b692a57
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:16.996207 [ 12459 ] {2cecf4e7-2064-4656-8035-0a060c27a9f9} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toLastDayOfWeek is allowed only when the 1st argument is DateTime or DateTime64.: In scope SELECT toLastDayOfWeek(toDate(16433), 1, 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionCustomWeek<DB::ToLastDayOfWeekImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool, bool) const @ 0x000000001b69adf8
4. DB::FunctionCustomWeekToDateOrDate32<DB::ToLastDayOfWeekImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b692a57
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:17.344849 [ 7099 ] {bc664921-0cdc-46d7-b270-88a070cedcd9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54146) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:17.349150 [ 7099 ] {bc664921-0cdc-46d7-b270-88a070cedcd9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:19.386100 [ 3764 ] {} <Error> void DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ZooKeeperLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::flushImpl(std::vector<DB::ZooKeeperLogElement, std::allocator<DB::ZooKeeperLogElement>> const&, unsigned long) @ 0x000000003608c8ea
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ZooKeeperLogElement>::savingThreadFunction() @ 0x0000000036089048
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ZooKeeperLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43a1a7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43a07f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:19.386096 [ 7099 ] {9d5f8d84-1597-44e8-b8ad-7e72805a4501} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54156) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:19.386097 [ 12459 ] {773dfbde-2014-4e32-bc3f-b58ae356d9d7} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toStartOfMonth is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toStartOfMonth(toDate(16433), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toStartOfMonth(toDate(16433), 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToStartOfMonthImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b7eead4
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToStartOfMonthImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b7e770e
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:19.387555 [ 12459 ] {773dfbde-2014-4e32-bc3f-b58ae356d9d7} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toStartOfMonth is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toStartOfMonth(toDate(16433), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToStartOfMonthImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b7eead4
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToStartOfMonthImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b7e770e
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:19.389913 [ 7099 ] {9d5f8d84-1597-44e8-b8ad-7e72805a4501} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:20.657831 [ 12459 ] {3f7e60a9-d87d-4a2c-8b7a-e64a2fc97301} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toStartOfQuarter is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toStartOfQuarter(toDate(16343), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toStartOfQuarter(toDate(16343), 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToStartOfQuarterImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b7f9db4
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToStartOfQuarterImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b7f138e
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:20.659376 [ 12459 ] {3f7e60a9-d87d-4a2c-8b7a-e64a2fc97301} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toStartOfQuarter is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toStartOfQuarter(toDate(16343), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToStartOfQuarterImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b7f9db4
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToStartOfQuarterImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b7f138e
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:21.622161 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/hcu/rlczmjgkzmofcyyvfpeebcujvikbb, total size 201326592, count 201326592, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f1fdb0a4-d12f-4eed-bb61-ee8d3959e4f7, upload has finished false.
2024.11.04 23:24:21.622345 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/hcu/rlczmjgkzmofcyyvfpeebcujvikbb, total size 201326592, count 201326592, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f1fdb0a4-d12f-4eed-bb61-ee8d3959e4f7, upload has finished false
2024.11.04 23:24:21.651037 [ 3697 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/hcu/rlczmjgkzmofcyyvfpeebcujvikbb, total size 201326592, count 201326592, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id f1fdb0a4-d12f-4eed-bb61-ee8d3959e4f7, upload has finished false
2024.11.04 23:24:22.248947 [ 12459 ] {d30dd39e-90b7-4990-80d0-d0f5b752191f} <Error> executeQuery: Code: 43. DB::Exception: The timezone argument of function toStartOfYear is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toStartOfYear(toDate(16433), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toStartOfYear(toDate(16433), 'Asia/Istanbul');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToStartOfYearImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b828414
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToStartOfYearImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b81ffce
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:22.249103 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 3 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:22.249096 [ 7099 ] {f8152fd3-e470-44ed-a901-6eef1d0bd4d8} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54180) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:22.250399 [ 12459 ] {d30dd39e-90b7-4990-80d0-d0f5b752191f} <Error> TCPHandler: Code: 43. DB::Exception: The timezone argument of function toStartOfYear is allowed only when the 1st argument has the type DateTime or DateTime64: In scope SELECT toStartOfYear(toDate(16433), 'Asia/Istanbul'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x0000000007c64995
3. DB::IFunctionDateOrDateTime<DB::ToStartOfYearImpl>::checkArguments(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&, bool) const @ 0x000000001b828414
4. DB::FunctionDateOrDateTimeToDateOrDate32<DB::ToStartOfYearImpl>::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000001b81ffce
5. DB::FunctionToOverloadResolverAdaptor::getReturnTypeImpl(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x0000000008edc83f
6. ./build_docker/./src/Functions/IFunction.cpp:522: DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd58329
7. ./build_docker/./src/Functions/IFunction.cpp:468: DB::IFunctionOverloadResolver::getReturnType(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd575b0
8. ./build_docker/./src/Functions/IFunction.cpp:483: DB::IFunctionOverloadResolver::build(std::vector<DB::ColumnWithTypeAndName, std::allocator<DB::ColumnWithTypeAndName>> const&) const @ 0x000000002fd5924d
9. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x000000003481900e
10. ./contrib/llvm-project/libcxx/include/vector:543: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347ba4c2
11. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x00000000347b7788
12. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000347d8583
13. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000034798fee
14. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>&, std::shared_ptr<DB::Context const>) @ 0x000000003478f330
15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003477d544
16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x0000000035734f20
17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x000000003572bfe5
18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: std::__unique_if<DB::InterpreterSelectQueryAnalyzer>::__unique_single std::make_unique[abi:v15000]<DB::InterpreterSelectQueryAnalyzer, std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&>(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context> const&, DB::SelectQueryOptions const&) @ 0x0000000035737fb5
19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:270: std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> std::__function::__policy_invoker<std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>::__call_impl<std::__function::__default_alloc_func<DB::registerInterpreterSelectQueryAnalyzer(DB::InterpreterFactory&)::$_0, std::unique_ptr<DB::IInterpreter, std::default_delete<DB::IInterpreter>> (DB::InterpreterFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::InterpreterFactory::Arguments const&) @ 0x0000000035737713
20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003559fa84
21. ./build_docker/./src/Interpreters/executeQuery.cpp:1133: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659bfc0
22. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
24. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
28. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
29. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850

2024.11.04 23:24:22.253437 [ 7099 ] {f8152fd3-e470-44ed-a901-6eef1d0bd4d8} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:22.267792 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:22.728424 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:22.811541 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:23.340680 [ 7099 ] {c93d1db1-2824-4f74-981e-6ee94e499a00} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54182) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:23.343833 [ 7099 ] {c93d1db1-2824-4f74-981e-6ee94e499a00} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:26.109924 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:26.402951 [ 12459 ] {3db203fb-a9c1-445b-a6ca-6637120df0e2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1011). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:38342) (comment: 00189_time_zones_long.sql) (in query: SELECT toString(toStartOfMinute(toDateTime(1549483055), 'Europe/Paris'), 'Europe/Paris');), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:26.404943 [ 12459 ] {3db203fb-a9c1-445b-a6ca-6637120df0e2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1011). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:28.051803 [ 2935 ] {b4a370b7-960c-4370-a351-1f878069e830} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60990) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:28.055744 [ 2935 ] {b4a370b7-960c-4370-a351-1f878069e830} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:29.006251 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:30.132586 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/koz/agsqecdhvrwytapgniokiikjvpfht, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 595f1cf4-10d8-4cfb-a504-7f19d801bbad, upload has finished false.
2024.11.04 23:24:30.132824 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/koz/agsqecdhvrwytapgniokiikjvpfht, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 595f1cf4-10d8-4cfb-a504-7f19d801bbad, upload has finished false
2024.11.04 23:24:30.175209 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/koz/agsqecdhvrwytapgniokiikjvpfht, total size 184549376, count 184549376, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 595f1cf4-10d8-4cfb-a504-7f19d801bbad, upload has finished false
2024.11.04 23:24:30.181496 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 48 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:30.274163 [ 7099 ] {326e697e-c54f-4ada-b3f9-42d90f1643bb} <Error> executeQuery: Code: 48. DB::Exception: JOIN ON constant supported only with join algorithm 'hash'. (NOT_IMPLEMENTED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32790) (comment: 02973_analyzer_join_use_nulls_column_not_found.sql) (in query: WITH a AS ( SELECT 0 AS key, 'a' AS acol ), b AS ( SELECT 2 AS key ) SELECT a.key FROM b LEFT JOIN a ON 1 LEFT JOIN a AS a1 ON 1 ;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ab79
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:24:30.276752 [ 7099 ] {326e697e-c54f-4ada-b3f9-42d90f1643bb} <Error> TCPHandler: Code: 48. DB::Exception: JOIN ON constant supported only with join algorithm 'hash'. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x0000000007c8938d
3. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr<DB::TableJoin>&, std::shared_ptr<DB::IQueryTreeNode> const&, DB::Block const&, DB::Block const&, std::shared_ptr<DB::PlannerContext> const&) @ 0x000000003586ab79
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr<DB::IQueryTreeNode> const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x0000000035817497
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357dfe7d
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:24:30.956594 [ 7099 ] {428a3711-d2e1-4a6d-ae18-68973bbc3d8f} <Error> executeQuery: Code: 57. DB::Exception: Table test_13.t already exists. (TABLE_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32814) (comment: 02910_replicated_merge_parameters_must_consistent.sql) (in query: CREATE TABLE t (`id` UInt64, `val` String, `legacy_ver` UInt64) ENGINE = ReplicatedReplacingMergeTree('/tables/{database}/t/', 'r1', legacy_ver) ORDER BY id SETTINGS index_granularity = 24922, min_bytes_for_wide_part = 1073741824, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = true, max_file_name_length = 0, min_bytes_for_full_part_storage = 253313315, compact_parts_max_bytes_to_buffer = 214462004, compact_parts_max_granules_to_buffer = 162, compact_parts_merge_max_bytes_to_prefetch_part = 5918838, merge_max_block_size = 17935, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 712079, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 10645475097, index_granularity_bytes = 22307506, concurrent_part_removal_threshold = 32, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = false, marks_compress_block_size = 81285, primary_key_compress_block_size = 51650), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:24:30.958245 [ 7099 ] {428a3711-d2e1-4a6d-ae18-68973bbc3d8f} <Error> TCPHandler: Code: 57. DB::Exception: Table test_13.t already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:24:31.114237 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f35a2ef5-e44d-4e42-94b5-c841517c6ce9::202411_1_116_29}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:31.427720 [ 7099 ] {a5025c68-0165-499d-91dc-11f3b317338e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32820) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:31.432355 [ 7099 ] {a5025c68-0165-499d-91dc-11f3b317338e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:32.600825 [ 2935 ] {f139ad93-c1fc-4ec2-aa01-491f59ac80b7} <Error> executeQuery: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32846) (in query: select _path,_file from file('nonexist.txt', 'CSV', 'val1 char')), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:24:32.602641 [ 2935 ] {f139ad93-c1fc-4ec2-aa01-491f59ac80b7} <Error> TCPHandler: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x0000000007cc47f5
3. ./build_docker/./src/Storages/StorageFile.cpp:1573: DB::StorageFile::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000037cce882
4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x00000000357ff9c5
5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1673: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x00000000357db4b4
6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000003574abd5
7. ./build_docker/./src/Planner/Planner.cpp:1244: DB::Planner::buildQueryPlanIfNeeded() @ 0x000000003573e451
8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000035737032
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659c79f
10. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
16. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
17. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
18. ? @ 0x00007fe94d0adac3
19. ? @ 0x00007fe94d13f850

2024.11.04 23:24:32.807819 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:33.982843 [ 7099 ] {01ef7d15-fe8e-47b0-9a35-7467fcb38e9e} <Error> executeQuery: Code: 636. DB::Exception: The table structure cannot be extracted from a Parquet format file. Error:
Code: 50. DB::Exception: Unsupported Parquet type 'null' of an input column 'error_message'. If it happens during schema inference and you want to skip columns with unsupported types, you can enable setting input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference. (UNKNOWN_TYPE) (version 24.3.12.76.altinitystable (altinity build)).
You can specify the structure manually: (in file/uri /var/lib/clickhouse/user_files/test_02245.parquet). (CANNOT_EXTRACT_TABLE_STRUCTURE) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32850) (comment: 02245_parquet_skip_unknown_type.sh) (in query: desc file(test_02245.parquet)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&) @ 0x000000001ac45ed5
3. ./build_docker/./src/Formats/ReadSchemaUtils.cpp:279: DB::readSchemaFromFormatImpl(std::optional<String>, std::optional<DB::FormatSettings> const&, DB::IReadBufferIterator&, std::shared_ptr<DB::Context const> const&) @ 0x000000003a5cf2a1
4. ./build_docker/./src/Formats/ReadSchemaUtils.cpp:0: DB::readSchemaFromFormat(String const&, std::optional<DB::FormatSettings> const&, DB::IReadBufferIterator&, std::shared_ptr<DB::Context const> const&) @ 0x000000003a5d2374
5. ./build_docker/./src/Storages/StorageFile.cpp:943: DB::StorageFile::getTableStructureAndFormatFromFileImpl(std::optional<String>, std::vector<String, std::allocator<String>> const&, String const&, std::optional<DB::FormatSettings> const&, std::shared_ptr<DB::Context const> const&, std::optional<DB::StorageFile::ArchiveInfo> const&) @ 0x0000000037ca7b2f
6. ./build_docker/./src/Storages/StorageFile.cpp:0: DB::StorageFile::getTableStructureFromFile(String const&, std::vector<String, std::allocator<String>> const&, String const&, std::optional<DB::FormatSettings> const&, std::shared_ptr<DB::Context const> const&, std::optional<DB::StorageFile::ArchiveInfo> const&) @ 0x0000000037ca9233
7. ./contrib/llvm-project/libcxx/include/optional:260: DB::TableFunctionFile::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4f668a
8. ./contrib/boost/boost/multi_index_container.hpp:640: DB::InterpreterDescribeQuery::fillColumnsFromTableFunction(DB::ASTTableExpression const&) @ 0x00000000366f8405
9. ./build_docker/./src/Interpreters/InterpreterDescribeQuery.cpp:0: DB::InterpreterDescribeQuery::execute() @ 0x00000000366f5365
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:24:33.982857 [ 2935 ] {db3ad0ab-8644-4284-8d53-0bdd24b84269} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53812) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:33.985577 [ 7099 ] {01ef7d15-fe8e-47b0-9a35-7467fcb38e9e} <Error> TCPHandler: Code: 636. DB::Exception: The table structure cannot be extracted from a Parquet format file. Error:
Code: 50. DB::Exception: Unsupported Parquet type 'null' of an input column 'error_message'. If it happens during schema inference and you want to skip columns with unsupported types, you can enable setting input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference. (UNKNOWN_TYPE) (version 24.3.12.76.altinitystable (altinity build)).
You can specify the structure manually: (in file/uri /var/lib/clickhouse/user_files/test_02245.parquet). (CANNOT_EXTRACT_TABLE_STRUCTURE), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String&, String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String&>::type>, String&, String&) @ 0x000000001ac45ed5
3. ./build_docker/./src/Formats/ReadSchemaUtils.cpp:279: DB::readSchemaFromFormatImpl(std::optional<String>, std::optional<DB::FormatSettings> const&, DB::IReadBufferIterator&, std::shared_ptr<DB::Context const> const&) @ 0x000000003a5cf2a1
4. ./build_docker/./src/Formats/ReadSchemaUtils.cpp:0: DB::readSchemaFromFormat(String const&, std::optional<DB::FormatSettings> const&, DB::IReadBufferIterator&, std::shared_ptr<DB::Context const> const&) @ 0x000000003a5d2374
5. ./build_docker/./src/Storages/StorageFile.cpp:943: DB::StorageFile::getTableStructureAndFormatFromFileImpl(std::optional<String>, std::vector<String, std::allocator<String>> const&, String const&, std::optional<DB::FormatSettings> const&, std::shared_ptr<DB::Context const> const&, std::optional<DB::StorageFile::ArchiveInfo> const&) @ 0x0000000037ca7b2f
6. ./build_docker/./src/Storages/StorageFile.cpp:0: DB::StorageFile::getTableStructureFromFile(String const&, std::vector<String, std::allocator<String>> const&, String const&, std::optional<DB::FormatSettings> const&, std::shared_ptr<DB::Context const> const&, std::optional<DB::StorageFile::ArchiveInfo> const&) @ 0x0000000037ca9233
7. ./contrib/llvm-project/libcxx/include/optional:260: DB::TableFunctionFile::getActualTableStructure(std::shared_ptr<DB::Context const>, bool) const @ 0x000000002f4f668a
8. ./contrib/boost/boost/multi_index_container.hpp:640: DB::InterpreterDescribeQuery::fillColumnsFromTableFunction(DB::ASTTableExpression const&) @ 0x00000000366f8405
9. ./build_docker/./src/Interpreters/InterpreterDescribeQuery.cpp:0: DB::InterpreterDescribeQuery::execute() @ 0x00000000366f5365
10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
11. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
13. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
17. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
18. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
19. ? @ 0x00007fe94d0adac3
20. ? @ 0x00007fe94d13f850

2024.11.04 23:24:33.986639 [ 2935 ] {db3ad0ab-8644-4284-8d53-0bdd24b84269} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:35.296144 [ 2935 ] {9994d77e-3162-42e7-b487-069e42417902} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53838) (comment: 02245_parquet_skip_unknown_type.sh) (in query: select count(*) from file(test_02245.parquet) settings input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference=1), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:35.302773 [ 2935 ] {9994d77e-3162-42e7-b487-069e42417902} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:36.100094 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:36.668484 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:24:36.668685 [ 3734 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:24:36.869747 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/kqm/esrjuahxfgqchbohayqzndskpupgs, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2de83e0c-b87a-417c-8ec7-d5ea60fe713a, upload has finished false.
2024.11.04 23:24:36.869954 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/kqm/esrjuahxfgqchbohayqzndskpupgs, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2de83e0c-b87a-417c-8ec7-d5ea60fe713a, upload has finished false
2024.11.04 23:24:37.000583 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/kqm/esrjuahxfgqchbohayqzndskpupgs, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2de83e0c-b87a-417c-8ec7-d5ea60fe713a, upload has finished false
2024.11.04 23:24:37.004942 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 70 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:37.275624 [ 7099 ] {07498ef6-9afc-41fc-ac32-129704e823c7} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53858) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:37.279009 [ 7099 ] {07498ef6-9afc-41fc-ac32-129704e823c7} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:38.521961 [ 3948 ] {18afc2f1-2559-4ce2-b9b8-c8551965e6a0} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53900) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: SELECT count() FROM test FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:38.525670 [ 3948 ] {18afc2f1-2559-4ce2-b9b8-c8551965e6a0} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:38.892048 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:40.397034 [ 13542 ] {09b6b897-1dde-4f86-abbd-eda7020f9103} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53968) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:40.402379 [ 13542 ] {09b6b897-1dde-4f86-abbd-eda7020f9103} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:40.489581 [ 2935 ] {f96ed291-d620-4864-a771-69b2753b935b} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:53946) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: OPTIMIZE TABLE test FINAL), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./src/Storages/MergeTree/MergePlainMergeTreeTask.h:104: DB::StorageMergeTree::merge(bool, String const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::MergeTreeTransaction> const&, String&, bool) @ 0x0000000039d39606
27. ./build_docker/./src/Storages/StorageMergeTree.cpp:1587: DB::StorageMergeTree::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x0000000039d50614
28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterOptimizeQuery::execute() @ 0x00000000366d6490
29. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
30. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
31. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28

2024.11.04 23:24:40.492535 [ 2935 ] {f96ed291-d620-4864-a771-69b2753b935b} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./src/Storages/MergeTree/MergePlainMergeTreeTask.h:104: DB::StorageMergeTree::merge(bool, String const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::MergeTreeTransaction> const&, String&, bool) @ 0x0000000039d39606
27. ./build_docker/./src/Storages/StorageMergeTree.cpp:1587: DB::StorageMergeTree::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x0000000039d50614
28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterOptimizeQuery::execute() @ 0x00000000366d6490
29. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
30. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
31. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28

2024.11.04 23:24:40.497534 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:40.564765 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 88 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:41.294958 [ 3948 ] {6965772b-a595-4027-9ba6-edfbabe2399e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54000) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: SELECT count() FROM test FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:41.298601 [ 3948 ] {6965772b-a595-4027-9ba6-edfbabe2399e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:41.353075 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=1): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:41.462079 [ 7099 ] {cdf497f9-8326-4bad-be11-5582ad007c47} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54014) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:41.466199 [ 7099 ] {cdf497f9-8326-4bad-be11-5582ad007c47} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=22, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:41.993938 [ 2935 ] {851d85a6-c70c-4741-8751-6439badca6c5} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54032) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: SELECT count() FROM test FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:41.998138 [ 2935 ] {851d85a6-c70c-4741-8751-6439badca6c5} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:42.962404 [ 7099 ] {342599e8-a071-4bc2-b511-be462ade0690} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54056) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:42.962432 [ 3948 ] {91cb3f08-3ff3-4ef1-8433-cdd2519001f4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54044) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: INSERT INTO test SELECT rand() FROM numbers(1000)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
25. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:24:42.966447 [ 7099 ] {342599e8-a071-4bc2-b511-be462ade0690} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:43.043676 [ 3948 ] {91cb3f08-3ff3-4ef1-8433-cdd2519001f4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c627
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:410: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6688c4
24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
25. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::CompletedPipelineExecutor::execute()::$_0>(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6656be
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850

2024.11.04 23:24:44.247631 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:44.247720 [ 3948 ] {76a38396-e5d8-41af-9e73-f85cdf4cb6fe} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54076) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: SELECT count() FROM test FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:44.253047 [ 3948 ] {76a38396-e5d8-41af-9e73-f85cdf4cb6fe} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:44.265565 [ 2935 ] {ad87861b-dc8c-4044-950b-b2df70a52645} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54072) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: OPTIMIZE TABLE test FINAL), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./src/Storages/MergeTree/MergePlainMergeTreeTask.h:104: DB::StorageMergeTree::merge(bool, String const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::MergeTreeTransaction> const&, String&, bool) @ 0x0000000039d39606
20. ./build_docker/./src/Storages/StorageMergeTree.cpp:1587: DB::StorageMergeTree::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x0000000039d50614
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterOptimizeQuery::execute() @ 0x00000000366d6490
22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:24:44.267855 [ 2935 ] {ad87861b-dc8c-4044-950b-b2df70a52645} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988fff2
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./src/Storages/MergeTree/MergePlainMergeTreeTask.h:104: DB::StorageMergeTree::merge(bool, String const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::MergeTreeTransaction> const&, String&, bool) @ 0x0000000039d39606
20. ./build_docker/./src/Storages/StorageMergeTree.cpp:1587: DB::StorageMergeTree::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x0000000039d50614
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterOptimizeQuery::execute() @ 0x00000000366d6490
22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:24:44.377772 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 28 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:44.856196 [ 2935 ] {41551b57-3547-47fc-8177-a3f2ee100459} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44322) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: SELECT count() FROM test FORMAT Null), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:44.861877 [ 2935 ] {41551b57-3547-47fc-8177-a3f2ee100459} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:45.156264 [ 7099 ] {5e0ce678-927b-4e26-8d38-fb37fa289f2d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44342) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:45.161077 [ 7099 ] {5e0ce678-927b-4e26-8d38-fb37fa289f2d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:45.174832 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 7 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:45.456822 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:45.506143 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:45.850086 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:46.038048 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:46.130501 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:46.368056 [ 7099 ] {8a6607b8-d028-4898-a0c1-141a1e526f63} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44398) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:46.371969 [ 7099 ] {8a6607b8-d028-4898-a0c1-141a1e526f63} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:46.548406 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=1): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_6_6_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:46.847372 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:47.315388 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:48.011984 [ 3708 ] {0d747137-4257-4328-a24a-e6d037fd9eff::all_17_17_0_18} <Error> MutatePlainMergeTreeTask: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:49.552603 [ 3785 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:49.552755 [ 2935 ] {dc32c5a4-d8df-45c4-80af-8424bce4e193} <Error> executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_18.txt' with part 'all_17_17_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44482) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: ALTER TABLE test MODIFY COLUMN x UInt8;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x0000000039d143c5
6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:24:49.901382 [ 7099 ] {11e851d8-420a-4e9b-b84a-aa284aa6548a} <Error> executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:44490). (NETWORK_ERROR) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44490) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: INSERT INTO test SELECT rand() FROM numbers(1000)), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProfileEvents() @ 0x000000003a50e93e
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d72f6
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:49.901459 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f7fd
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:49.901655 [ 2935 ] {dc32c5a4-d8df-45c4-80af-8424bce4e193} <Error> TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_18.txt' with part 'all_17_17_0' reason: 'Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x000000003966eac4
3. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x000000003966e385
4. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000039d1ffb6
5. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::alter(DB::AlterCommands const&, std::shared_ptr<DB::Context const>, std::unique_lock<std::timed_mutex>&) @ 0x0000000039d143c5
6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:0: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x00000000345d4afc
7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:71: DB::InterpreterAlterQuery::execute() @ 0x00000000345cb930
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
9. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:24:49.901742 [ 3708 ] {0d747137-4257-4328-a24a-e6d037fd9eff::all_17_17_0_18} <Error> virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationNullable::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c6ace1
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:0: DB::ReadFromPart::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x0000000039737603
23. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:0: DB::ISourceStep::updatePipeline(std::vector<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>, std::allocator<std::unique_ptr<DB::QueryPipelineBuilder, std::default_delete<DB::QueryPipelineBuilder>>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000003b5fb343
24. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:188: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000003b668231
25. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:1294: DB::MutationsInterpreter::addStreamsForLaterStages(std::vector<DB::MutationsInterpreter::Stage, std::allocator<DB::MutationsInterpreter::Stage>> const&, DB::QueryPlan&) const @ 0x0000000035a2bed4
26. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:0: DB::MutationsInterpreter::execute() @ 0x0000000035a35869
27. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::prepare() @ 0x00000000398d04ba
28. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:2000: DB::MutateTask::execute() @ 0x00000000398ca8e5
29. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:0: DB::MutatePlainMergeTreeTask::executeStep() @ 0x0000000039dd7d22
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:49.901758 [ 13542 ] {525108ea-ee22-477a-b4af-010ceb5fafa3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44450) (comment: 01002_alter_nullable_adaptive_granularity_long.sh) (in query: OPTIMIZE TABLE test FINAL), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./src/Storages/MergeTree/MergePlainMergeTreeTask.h:104: DB::StorageMergeTree::merge(bool, String const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::MergeTreeTransaction> const&, String&, bool) @ 0x0000000039d39606
20. ./build_docker/./src/Storages/StorageMergeTree.cpp:1587: DB::StorageMergeTree::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x0000000039d50614
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterOptimizeQuery::execute() @ 0x00000000366d6490
22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:24:49.901784 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:49.901810 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:49.902425 [ 2935 ] {dc32c5a4-d8df-45c4-80af-8424bce4e193} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:24:49.903380 [ 7099 ] {11e851d8-420a-4e9b-b84a-aa284aa6548a} <Error> TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:44490). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000003a525ebe
5. ./src/Core/Block.h:25: DB::TCPHandler::sendLogs() @ 0x000000003a4fd92b
6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::TCPHandler::runImpl() @ 0x000000003a4d7721
7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
11. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
12. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
13. ? @ 0x00007fe94d0adac3
14. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:49.903681 [ 7099 ] {11e851d8-420a-4e9b-b84a-aa284aa6548a} <Error> TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:44490). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: DB::TCPHandler::sendProfileEvents() @ 0x000000003a50e93e
5. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d72f6
6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
7. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
9. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
10. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
11. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:49.904200 [ 7099 ] {11e851d8-420a-4e9b-b84a-aa284aa6548a} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:24:49.904720 [ 13542 ] {525108ea-ee22-477a-b4af-010ceb5fafa3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./src/Storages/MergeTree/MergePlainMergeTreeTask.h:104: DB::StorageMergeTree::merge(bool, String const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::MergeTreeTransaction> const&, String&, bool) @ 0x0000000039d39606
20. ./build_docker/./src/Storages/StorageMergeTree.cpp:1587: DB::StorageMergeTree::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x0000000039d50614
21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterOptimizeQuery::execute() @ 0x00000000366d6490
22. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
23. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
25. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
29. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
30. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
31. ? @ 0x00007fe94d0adac3

2024.11.04 23:24:49.905583 [ 13542 ] {525108ea-ee22-477a-b4af-010ceb5fafa3} <Warning> TCPHandler: Client has gone away.
2024.11.04 23:24:50.627391 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_142_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=70). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:52.463276 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:52.463321 [ 2935 ] {30068cea-ef08-49ba-97e3-5a249bf111f2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44548) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:52.467246 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_142_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=36). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
16. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
17. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
19. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
20. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
21. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
27. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
31. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:52.468882 [ 2935 ] {30068cea-ef08-49ba-97e3-5a249bf111f2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:52.508154 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {33f0dc8f-1542-43c0-a136-e415d11993b4::202411_1_122_24}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:53.558716 [ 2935 ] {ef968598-dc9d-4e8d-8c91-c1d8856eebef} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50426) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:53.563133 [ 2935 ] {ef968598-dc9d-4e8d-8c91-c1d8856eebef} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:53.725243 [ 7099 ] {9b795707-e40a-4378-8dd3-3df6fe35cd5c} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:44558) (comment: 01182_materialized_view_different_structure.sql) (in query: SELECT sum(value) FROM test_mv;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:53.729655 [ 7099 ] {9b795707-e40a-4378-8dd3-3df6fe35cd5c} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:54.646703 [ 2935 ] {1cd72100-e91a-49aa-bd9c-105c6dd9c178} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50434) (comment: 03022_highlight_digit_groups.sql) (in query: SELECT exp10(number) + exp10(-number) FROM numbers(10) FORMAT PrettySpace SETTINGS output_format_pretty_color = 1;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:54.649463 [ 2935 ] {1cd72100-e91a-49aa-bd9c-105c6dd9c178} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:55.212106 [ 2935 ] {12a388a2-bf4e-4ab2-8727-c40392a52cfb} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50450) (comment: 02477_is_null_parser.sql) (in query: EXPLAIN SYNTAX SELECT 1 IS NULL + 1 IS NOT NULL;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:55.214653 [ 2935 ] {12a388a2-bf4e-4ab2-8727-c40392a52cfb} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1182, jobs=1010). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35032e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f34077c
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f3415f4
6. ./src/Common/ThreadPool.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000003a6a523c
7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000003a6a6205
8. ./build_docker/./src/Server/TCPHandler.cpp:1023: DB::TCPHandler::processOrdinaryQuery() @ 0x000000003a4f9a59
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000003a4d6ff7
10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
14. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
15. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
16. ? @ 0x00007fe94d0adac3
17. ? @ 0x00007fe94d13f850

2024.11.04 23:24:55.262601 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_142_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=2): (while reading column used_aggregate_functions): (while reading from part /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/202411_1_122_21/ in table system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:55.385267 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_141_30}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=4): (while reading column timestamp_ns): (while reading from part /var/lib/clickhouse/disks/s3/store/f42/f421370c-92f1-4a0e-839e-bdd872778e02/202411_1_135_29/ in table system.trace_log (f421370c-92f1-4a0e-839e-bdd872778e02) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:55.717448 [ 2935 ] {9d29f575-d12f-4ba0-b475-bcb16fff9148} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50464) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:55.721907 [ 2935 ] {9d29f575-d12f-4ba0-b475-bcb16fff9148} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:56.063699 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/dst/cejnkajbdvjjqssrouejitmdqjntu, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2694a3d3-9881-4ee5-bee1-64696fa9a92f, upload has finished false.
2024.11.04 23:24:56.063911 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/dst/cejnkajbdvjjqssrouejitmdqjntu, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2694a3d3-9881-4ee5-bee1-64696fa9a92f, upload has finished false
2024.11.04 23:24:56.085502 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/dst/cejnkajbdvjjqssrouejitmdqjntu, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 2694a3d3-9881-4ee5-bee1-64696fa9a92f, upload has finished false
2024.11.04 23:24:56.089573 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 73 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:56.243800 [ 7099 ] {6c08fc6d-2ffe-4196-949b-a4d9a1a5421f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50472) (comment: 02111_with_fill_no_rows.sql) (in query: SELECT toYear(d) AS y, count() FROM ( SELECT today() AS d WHERE 0) GROUP BY y ORDER BY y ASC WITH FILL FROM 2019;), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:56.255963 [ 7099 ] {6c08fc6d-2ffe-4196-949b-a4d9a1a5421f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:24:56.528902 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_142_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:56.765267 [ 2935 ] {98645e89-98b9-4c66-821b-92508536aa26} <Error> executeQuery: Code: 57. DB::Exception: Table test_13.t already exists. (TABLE_ALREADY_EXISTS) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50474) (comment: 02514_bad_index_granularity.sql) (in query: CREATE TABLE t (`id` Int64, `d` String, `p` Map(String, String)) ENGINE = ReplacingMergeTree ORDER BY id SETTINGS index_granularity = 0, min_bytes_for_wide_part = 1065069806, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = true, max_file_name_length = 105, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 53453745, compact_parts_max_granules_to_buffer = 1, compact_parts_merge_max_bytes_to_prefetch_part = 4801883, merge_max_block_size = 6176, old_parts_lifetime = 30., prefer_fetch_merged_part_size_threshold = 232716864, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 10737418240, index_granularity_bytes = 17842370, concurrent_part_removal_threshold = 91, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 49449, primary_key_compress_block_size = 73209), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:24:56.766565 [ 2935 ] {98645e89-98b9-4c66-821b-92508536aa26} <Error> TCPHandler: Code: 57. DB::Exception: Table test_13.t already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007c671d5
3. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1395: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x000000003466734d
4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000034643699
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1794: DB::InterpreterCreateQuery::execute() @ 0x0000000034673267
6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
7. ./build_docker/./src/Interpreters/executeQuery.cpp:1374: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000036590723
8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723: DB::TCPHandler::runImpl() @ 0x000000003a4d6b28
9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x000000003a52635c
10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
13. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
14. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
15. ? @ 0x00007fe94d0adac3
16. ? @ 0x00007fe94d13f850

2024.11.04 23:24:57.550273 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_141_30}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:45: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e0762
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:57.836838 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:57.929448 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_142_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=39). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:57.934838 [ 2935 ] {e54927dc-86b9-46b5-b252-b69d4b6d4f64} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50502) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:57.939267 [ 2935 ] {e54927dc-86b9-46b5-b252-b69d4b6d4f64} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:24:58.355569 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:58.675137 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:59.142307 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:24:59.450370 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:00.172662 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:00.538571 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:00.558940 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:01.771464 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
16. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
17. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
19. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
20. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
21. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
27. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
31. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:02.842924 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:03.551867 [ 2935 ] {1f6b5047-10df-43dd-baea-db3e1c3bca62} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40240) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:03.579663 [ 2935 ] {1f6b5047-10df-43dd-baea-db3e1c3bca62} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:04.365068 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=70). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c65901
14. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:641: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cfe010
15. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
16. ./build_docker/./src/DataTypes/Serializations/SerializationMap.cpp:0: DB::SerializationMap::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c63e21
17. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
18. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
19. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
20. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
23. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
24. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
25. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
29. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
30. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:04.670299 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:04.750787 [ 2935 ] {337c75fd-d765-4b3a-a18e-cc5753f5e0b1} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40248) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:04.754432 [ 2935 ] {337c75fd-d765-4b3a-a18e-cc5753f5e0b1} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=23, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:04.824976 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:05.396016 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:05.848545 [ 2935 ] {75fb85fb-93ef-4036-a32a-5196e4edd7fa} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40254) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:05.851779 [ 2935 ] {75fb85fb-93ef-4036-a32a-5196e4edd7fa} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:06.987306 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_230_260_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:07.436725 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_143_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=6): (while reading column client_version_patch): (while reading from part /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/202411_1_122_21/ in table system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:07.886617 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5902c0d6-2094-4687-afb1-23195ef9ef4b::202411_230_260_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:08.454472 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/wqd/vjwmuesxwemunonzlhtancbnlycjy, total size 369098752, count 369098752, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d9f81b13-7cfb-4a68-b191-9fd95b43f074, upload has finished false.
2024.11.04 23:25:08.454749 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/wqd/vjwmuesxwemunonzlhtancbnlycjy, total size 369098752, count 369098752, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d9f81b13-7cfb-4a68-b191-9fd95b43f074, upload has finished false
2024.11.04 23:25:08.510047 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/wqd/vjwmuesxwemunonzlhtancbnlycjy, total size 369098752, count 369098752, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d9f81b13-7cfb-4a68-b191-9fd95b43f074, upload has finished false
2024.11.04 23:25:08.518382 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 47 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:11.349308 [ 2935 ] {ba307fd3-4ea7-4cc2-83a9-0fdf4c81bd69} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40306) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:11.353240 [ 2935 ] {ba307fd3-4ea7-4cc2-83a9-0fdf4c81bd69} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:12.423183 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:12.465278 [ 2935 ] {3aebdefb-e4a6-4c1c-8f01-6669d052daf4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40308) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:12.469398 [ 2935 ] {3aebdefb-e4a6-4c1c-8f01-6669d052daf4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:13.174347 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:13.512446 [ 2935 ] {1bab6c02-3865-40fc-9262-5c46c44cea3f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50214) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:13.515682 [ 2935 ] {1bab6c02-3865-40fc-9262-5c46c44cea3f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=24, jobs=24). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:13.808822 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:14.215610 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:15.024070 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:25:15.024531 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:25:15.272170 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/dps/hdisvvipqtfxrgloegadibzdlzwja, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 84d7de3a-76a5-4748-bb08-a911ab93b317, upload has finished false.
2024.11.04 23:25:15.307905 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/dps/hdisvvipqtfxrgloegadibzdlzwja, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 84d7de3a-76a5-4748-bb08-a911ab93b317, upload has finished false
2024.11.04 23:25:15.329449 [ 3708 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/dps/hdisvvipqtfxrgloegadibzdlzwja, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 84d7de3a-76a5-4748-bb08-a911ab93b317, upload has finished false
2024.11.04 23:25:15.338880 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 8 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:16.996179 [ 2935 ] {e0b5b746-1bd0-4221-8fcb-e0789d718d63} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50234) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:16.999901 [ 2935 ] {e0b5b746-1bd0-4221-8fcb-e0789d718d63} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:17.411503 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 19 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:18.140835 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:18.300420 [ 2935 ] {ae840a7c-b1cf-4b4b-836a-76e7f45389c3} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50238) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:18.303985 [ 2935 ] {ae840a7c-b1cf-4b4b-836a-76e7f45389c3} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:19.323429 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:20.684114 [ 2935 ] {f9f22d3d-69f5-4518-a705-7afd4a2f314e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:50252) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:20.687939 [ 2935 ] {f9f22d3d-69f5-4518-a705-7afd4a2f314e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:21.468355 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_128_41}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:22.512683 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_128_41}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:24.255954 [ 3767 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:24.267238 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=29). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:25.280646 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:25.710612 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:25.765690 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/vqb/ssupntcqcjtlgmtawiubuiutytwqr, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id adac6fee-aa7c-409d-9cd6-c0fab2c86107, upload has finished false.
2024.11.04 23:25:25.765953 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/vqb/ssupntcqcjtlgmtawiubuiutytwqr, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id adac6fee-aa7c-409d-9cd6-c0fab2c86107, upload has finished false
2024.11.04 23:25:25.803143 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/vqb/ssupntcqcjtlgmtawiubuiutytwqr, total size 218103808, count 218103808, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id adac6fee-aa7c-409d-9cd6-c0fab2c86107, upload has finished false
2024.11.04 23:25:25.808817 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 23 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:25.886705 [ 3763 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:26.245240 [ 2935 ] {57fd565b-19d5-4205-b216-5e6b33203d50} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32846) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:26.249281 [ 2935 ] {57fd565b-19d5-4205-b216-5e6b33203d50} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:26.501789 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:26.940874 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:27.271829 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 25 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:27.309820 [ 2935 ] {1dadc295-beec-4cf2-97a4-95c71e8176e9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32852) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:27.313216 [ 2935 ] {1dadc295-beec-4cf2-97a4-95c71e8176e9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:27.349635 [ 3758 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:27.542937 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:28.622352 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:29.250358 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 28 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:29.562321 [ 2935 ] {f715dd97-ada5-4082-a03f-afff980706b4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32868) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:29.567462 [ 2935 ] {f715dd97-ada5-4082-a03f-afff980706b4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:30.688824 [ 2935 ] {bc74292d-368b-4c6a-91dd-9ab46c7e9372} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:32882) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:30.692207 [ 2935 ] {bc74292d-368b-4c6a-91dd-9ab46c7e9372} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:25:32.894316 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:33.475204 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:34.683812 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:35.113726 [ 2935 ] {6c96788c-3005-4e30-a9e9-fab0c30dd8dc} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37130) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:35.117329 [ 2935 ] {6c96788c-3005-4e30-a9e9-fab0c30dd8dc} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=18, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:35.253933 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:35.778225 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/mux/rnmhxvbmatpkyqiqsmnpuhpvuhkme, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 535838f6-ddf8-4c9e-a0af-6af1c5eb890f, upload has finished false.
2024.11.04 23:25:35.778484 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/mux/rnmhxvbmatpkyqiqsmnpuhpvuhkme, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 535838f6-ddf8-4c9e-a0af-6af1c5eb890f, upload has finished false
2024.11.04 23:25:35.801058 [ 3699 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/mux/rnmhxvbmatpkyqiqsmnpuhpvuhkme, total size 134217728, count 134217728, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 535838f6-ddf8-4c9e-a0af-6af1c5eb890f, upload has finished false
2024.11.04 23:25:35.821964 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 55 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:36.580840 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:37.950374 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:38.889794 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:41.829106 [ 2935 ] {13f4fa19-e85d-4d51-841b-2f591aedaf24} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:37188) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:41.832080 [ 2935 ] {13f4fa19-e85d-4d51-841b-2f591aedaf24} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:42.905879 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=19). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:43.625874 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/yta/ghslqikjeqohibbnwxrhwujyumyav, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d0d1f99b-6fd9-4537-9e72-7e78327d8bcc, upload has finished false.
2024.11.04 23:25:43.626220 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/yta/ghslqikjeqohibbnwxrhwujyumyav, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d0d1f99b-6fd9-4537-9e72-7e78327d8bcc, upload has finished false
2024.11.04 23:25:43.679104 [ 3703 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/yta/ghslqikjeqohibbnwxrhwujyumyav, total size 167772160, count 167772160, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id d0d1f99b-6fd9-4537-9e72-7e78327d8bcc, upload has finished false
2024.11.04 23:25:43.733640 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 39 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:43.951141 [ 2935 ] {eb799d47-0402-435a-b4a4-eedc816b14db} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58146) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:43.954843 [ 2935 ] {eb799d47-0402-435a-b4a4-eedc816b14db} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:44.438957 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:48.029819 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:25:48.029986 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:25:48.084640 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:298: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1cc1
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:48.650732 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationSparse.cpp:167: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cdd498
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:49.203756 [ 3703 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_143_31}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=2): (while reading column event_time): (while reading from part /var/lib/clickhouse/disks/s3/store/f42/f421370c-92f1-4a0e-839e-bdd872778e02/202411_1_141_30/ in table system.trace_log (f421370c-92f1-4a0e-839e-bdd872778e02) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:49.297358 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=30). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:49.626064 [ 2935 ] {a1911988-2283-455a-a35f-c34fba78dd28} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58184) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:49.631256 [ 2935 ] {a1911988-2283-455a-a35f-c34fba78dd28} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:50.099993 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/wxe/jopaeiazehquywfkizjtpstrksyqs, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 0a246beb-8951-498a-99be-b9ca76104f6e, upload has finished false.
2024.11.04 23:25:50.100159 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/wxe/jopaeiazehquywfkizjtpstrksyqs, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 0a246beb-8951-498a-99be-b9ca76104f6e, upload has finished false
2024.11.04 23:25:50.107480 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/wxe/jopaeiazehquywfkizjtpstrksyqs, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 0a246beb-8951-498a-99be-b9ca76104f6e, upload has finished false
2024.11.04 23:25:50.110838 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 19 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:50.220453 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:0: DB::MergeTreePartition::store(DB::Block const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&, DB::WriteSettings const&) const @ 0x0000000039690de3
13. ./build_docker/./src/Storages/MergeTree/MergeTreePartition.cpp:421: DB::MergeTreePartition::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x000000003968e74e
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:239: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988ca08
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:50.386796 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c65901
14. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:641: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cfe010
15. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
16. ./build_docker/./src/DataTypes/Serializations/SerializationMap.cpp:0: DB::SerializationMap::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c63e21
17. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
18. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
19. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
20. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
21. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
23. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
24. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
25. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
29. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
30. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:51.224074 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:51.523856 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=31). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:51.806961 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:51.882176 [ 2935 ] {7f524d46-badc-41f7-a572-6525cac8f08f} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:58212) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:51.887600 [ 2935 ] {7f524d46-badc-41f7-a572-6525cac8f08f} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=25, jobs=25). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:52.040177 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:52.556185 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 67 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:52.703955 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=12). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
16. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
17. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
19. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
20. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
21. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
27. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
31. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:53.007317 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:53.155683 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:53.816779 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:54.426774 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:55.122642 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=36). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/SerializationLowCardinality.cpp:59: DB::SerializationLowCardinality::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c3ddd9
13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
14. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
15. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
16. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
17. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
18. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
19. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
20. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
21. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
22. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
27. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
31. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:55.473678 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=88). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:55.811245 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=34). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:56.122337 [ 3709 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationSparse::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cde4dd
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:56.154933 [ 2935 ] {9fabb7b3-5422-4557-9358-a42a4dc7e87a} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60350) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:56.158327 [ 2935 ] {9fabb7b3-5422-4557-9358-a42a4dc7e87a} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:25:56.829648 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:57.193905 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:57.291115 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/gjf/lyddxkpjlnewetexcthxrmgjetnxa, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 93567057-ad9e-4ab1-85f3-733dc14d5800, upload has finished false.
2024.11.04 23:25:57.291305 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/gjf/lyddxkpjlnewetexcthxrmgjetnxa, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 93567057-ad9e-4ab1-85f3-733dc14d5800, upload has finished false
2024.11.04 23:25:57.311355 [ 3704 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/gjf/lyddxkpjlnewetexcthxrmgjetnxa, total size 100663296, count 100663296, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 93567057-ad9e-4ab1-85f3-733dc14d5800, upload has finished false
2024.11.04 23:25:57.315662 [ 3704 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 19 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:57.788169 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:58.108868 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=71). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:25:58.854290 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=34). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
15. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
16. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
19. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
20. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
26. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
28. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
30. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
31. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:01.831842 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=23). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:02.278257 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:02.664762 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:02.899979 [ 2935 ] {3de0ad82-481c-41c8-bf9a-6ab65d755543} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:60400) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:02.903911 [ 2935 ] {3de0ad82-481c-41c8-bf9a-6ab65d755543} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:03.388667 [ 3736 ] {} <Warning> system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b): Removing temporary directory /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/tmp_merge_202411_1_144_22/
2024.11.04 23:26:03.388854 [ 3736 ] {} <Warning> system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/tmp_merge_202411_1_144_22/
2024.11.04 23:26:03.550147 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:03.635028 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_144_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=104). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:04.121287 [ 2935 ] {d7f40556-88a2-48ed-bea6-35bbc1295d28} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49868) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:04.124929 [ 2935 ] {d7f40556-88a2-48ed-bea6-35bbc1295d28} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:05.225577 [ 2935 ] {69e35120-074b-4ccd-86b4-bf97fcf5b710} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49874) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:26:05.229870 [ 2935 ] {69e35120-074b-4ccd-86b4-bf97fcf5b710} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:26:05.818021 [ 3699 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_149_32}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=22). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:05.856903 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ced847ce-158f-4cce-b25e-de045462141e::202411_139_154_3}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
22. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
23. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
24. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
25. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
26. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
28. ? @ 0x00007fe94d0adac3
29. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:06.367815 [ 2935 ] {19106d9c-6619-4593-a22a-8a610224f7c4} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49882) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:26:06.371155 [ 2935 ] {19106d9c-6619-4593-a22a-8a610224f7c4} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=21, jobs=21). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:26:07.252826 [ 3783 ] {} <Error> void DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::ProcessorProfileLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::flushImpl(std::vector<DB::ProcessorProfileLogElement, std::allocator<DB::ProcessorProfileLogElement>> const&, unsigned long) @ 0x00000000360a22aa
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::ProcessorProfileLogElement>::savingThreadFunction() @ 0x000000003609ea08
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::ProcessorProfileLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43b4e7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43b3bf
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:07.425041 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/dlh/ncmpfjuioqqkivhfrzycpikoipluz, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b2996bbd-e5b5-4eef-9d02-8c37ce1a5c6e, upload has finished false.
2024.11.04 23:26:07.425306 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/dlh/ncmpfjuioqqkivhfrzycpikoipluz, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b2996bbd-e5b5-4eef-9d02-8c37ce1a5c6e, upload has finished false
2024.11.04 23:26:07.465421 [ 3701 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/dlh/ncmpfjuioqqkivhfrzycpikoipluz, total size 251658240, count 251658240, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id b2996bbd-e5b5-4eef-9d02-8c37ce1a5c6e, upload has finished false
2024.11.04 23:26:07.471177 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_5_5_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 72 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:08.342875 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {f421370c-92f1-4a0e-839e-bdd872778e02::202411_1_149_32}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp:47: DB::MergeTreeDataPartWriterOnDisk::Stream<false>::preFinalize() @ 0x00000000393e078b
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:645: DB::MergeTreeDataPartWriterWide::fillDataChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x000000003940040e
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:683: DB::MergeTreeDataPartWriterWide::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x0000000039402185
15. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
16. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
20. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
21. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
23. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
24. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
25. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
26. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
27. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
29. ? @ 0x00007fe94d0adac3
30. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:08.577103 [ 2935 ] {c3ce6ca3-5ece-4834-9906-f3ac704fbde2} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49898) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:08.580759 [ 2935 ] {c3ce6ca3-5ece-4834-9906-f3ac704fbde2} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=6, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:09.858566 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_1_1_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 79 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:10.616820 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:11.725697 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:12.896651 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:13.015317 [ 2935 ] {97b159e5-7d4f-49da-9550-b0f25af01b36} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:49940) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:13.018911 [ 2935 ] {97b159e5-7d4f-49da-9550-b0f25af01b36} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:14.130010 [ 2935 ] {8e8cf340-345a-44b1-b5b8-2b63e36d43da} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40564) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:14.133561 [ 2935 ] {8e8cf340-345a-44b1-b5b8-2b63e36d43da} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=20, jobs=20). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:17.541345 [ 2935 ] {77e11423-a1fd-4b13-a436-f3ff88a7ec3e} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40580) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:17.545255 [ 2935 ] {77e11423-a1fd-4b13-a436-f3ff88a7ec3e} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:20.312618 [ 3697 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:20.942185 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/ham/gffwnzoerrdmmjgwtgbklppeddjek, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e3d0c302-244f-4f87-91d9-93fad7c8c1be, upload has finished false.
2024.11.04 23:26:20.942424 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/ham/gffwnzoerrdmmjgwtgbklppeddjek, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e3d0c302-244f-4f87-91d9-93fad7c8c1be, upload has finished false
2024.11.04 23:26:20.986764 [ 3705 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/ham/gffwnzoerrdmmjgwtgbklppeddjek, total size 352321536, count 352321536, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e3d0c302-244f-4f87-91d9-93fad7c8c1be, upload has finished false
2024.11.04 23:26:20.991904 [ 3705 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_7_7_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:21.109413 [ 2935 ] {d75c74ed-95f8-4b42-8b75-2dcbb231ddce} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40606) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:21.114991 [ 2935 ] {d75c74ed-95f8-4b42-8b75-2dcbb231ddce} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:22.881564 [ 3701 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_113_143_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988dace
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:23.251384 [ 2935 ] {8fa850fd-ab99-4ca0-82d1-ebb25a1fd9d9} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:40628) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:23.254428 [ 2935 ] {8fa850fd-ab99-4ca0-82d1-ebb25a1fd9d9} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:23.264360 [ 3708 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {6a51ab79-088f-4d27-bb32-6b1f66ed39e5::202411_1_135_48}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007fe94d0adac3
28. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:24.393649 [ 2935 ] {7c6c032f-16cd-420e-931d-e07d1b1656d6} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34328) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:24.397652 [ 2935 ] {7c6c032f-16cd-420e-931d-e07d1b1656d6} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=16, jobs=16). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:26.066786 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:26:26.067008 [ 3737 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:26:26.095099 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/wud/teeuohyetygywclkzqnllfsljmegm, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e3cfc17a-a9da-4202-8878-90cf9dd14737, upload has finished false.
2024.11.04 23:26:26.095318 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/wud/teeuohyetygywclkzqnllfsljmegm, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e3cfc17a-a9da-4202-8878-90cf9dd14737, upload has finished false
2024.11.04 23:26:26.105406 [ 3700 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/wud/teeuohyetygywclkzqnllfsljmegm, total size 67108864, count 67108864, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id e3cfc17a-a9da-4202-8878-90cf9dd14737, upload has finished false
2024.11.04 23:26:26.110823 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 56 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:26.133611 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:26.452803 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:26.683658 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_28_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=18). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:28.804018 [ 2935 ] {22092f47-8896-48c6-bfcf-5a0ec7b69234} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34352) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:28.807679 [ 2935 ] {22092f47-8896-48c6-bfcf-5a0ec7b69234} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:29.858746 [ 2935 ] {886d0366-7459-477b-8c87-17c2bbc89964} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:34356) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:26:29.862689 [ 2935 ] {886d0366-7459-477b-8c87-17c2bbc89964} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: cannot allocate thread (threads=15, jobs=15). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f347af1
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./src/Processors/Executors/ExecutionThreadContext.h:58: DB::PipelineExecutor::executeSingleThread(unsigned long) @ 0x000000003a66c5c6
8. ./base/base/../base/scope_guard.h:50: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::PipelineExecutor::spawnThreads()::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x000000003a66d866
9. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
10. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
12. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
14. ? @ 0x00007fe94d0adac3
15. ? @ 0x00007fe94d13f850

2024.11.04 23:26:30.459280 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:31.094656 [ 3736 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:26:31.094817 [ 3736 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:26:31.146744 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/aek/qfynzuvuivpzwoynathqbfiopkeve, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c54eaa42-e594-4c22-910f-84346214f8f4, upload has finished false.
2024.11.04 23:26:31.146938 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/aek/qfynzuvuivpzwoynathqbfiopkeve, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c54eaa42-e594-4c22-910f-84346214f8f4, upload has finished false
2024.11.04 23:26:31.162871 [ 3694 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/aek/qfynzuvuivpzwoynathqbfiopkeve, total size 83886080, count 83886080, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id c54eaa42-e594-4c22-910f-84346214f8f4, upload has finished false
2024.11.04 23:26:31.167548 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 8 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:31.365116 [ 3702 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:32.234906 [ 3696 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:32.670081 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:32.877469 [ 3766 ] {} <Error> void DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::OpenTelemetrySpanLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::flushImpl(std::vector<DB::OpenTelemetrySpanLogElement, std::allocator<DB::OpenTelemetrySpanLogElement>> const&, unsigned long) @ 0x0000000035fdf68b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::OpenTelemetrySpanLogElement>::savingThreadFunction() @ 0x0000000035fdc069
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::OpenTelemetrySpanLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f430987
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43085f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:33.597042 [ 3762 ] {} <Error> void DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::AsynchronousMetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988f34c
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::flushImpl(std::vector<DB::AsynchronousMetricLogElement, std::allocator<DB::AsynchronousMetricLogElement>> const&, unsigned long) @ 0x0000000035f9e94b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::AsynchronousMetricLogElement>::savingThreadFunction() @ 0x0000000035f9b329
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::AsynchronousMetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42d4c7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42d39f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:33.781293 [ 3759 ] {} <Error> void DB::SystemLog<DB::PartLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::PartLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::flushImpl(std::vector<DB::PartLogElement, std::allocator<DB::PartLogElement>> const&, unsigned long) @ 0x0000000035ff520a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::PartLogElement>::savingThreadFunction() @ 0x0000000035ff1968
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::PartLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f431cc7
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f431b9f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007fe94d0adac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:35.269219 [ 2935 ] {d9ec587f-69e3-4a6d-9428-a837f494a57d} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:54100) (comment: 01057_window_view_event_tumble_to_strict_asc.sh) (in query: SELECT count(*) FROM dst), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:35.273321 [ 2935 ] {d9ec587f-69e3-4a6d-9428-a837f494a57d} <Error> TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::spawnThreads() @ 0x000000003a66cfdb
7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000003a6687d5
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000003a668328
9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0>(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003a6a845d
10. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
12. ? @ 0x00007fe94d0adac3
13. ? @ 0x00007fe94d13f850

2024.11.04 23:26:35.805704 [ 3700 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:574: DB::MergeTask::VerticalMergeStage::prepareVerticalMergeForOneColumn() const @ 0x000000003905b6ff
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::VerticalMergeStage::executeVerticalMergeForAllColumns() const @ 0x000000003906b243
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:829: DB::MergeTask::VerticalMergeStage::execute() @ 0x000000003906af84
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:36.311075 [ 3787 ] {} <Error> void DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::BlobStorageLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=100, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/string:1499: DB::MergeTreeDataPartWriterOnDisk::fillPrimaryIndexChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393d9a65
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:423: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c73
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::flushImpl(std::vector<DB::BlobStorageLogElement, std::allocator<DB::BlobStorageLogElement>> const&, unsigned long) @ 0x000000003613968a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::BlobStorageLogElement>::savingThreadFunction() @ 0x0000000036135de8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::BlobStorageLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f443667
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f44353f
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:36.411914 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Removing temporary directory /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:26:36.412038 [ 3743 ] {} <Warning> test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/tmp_merge_all_1_9_1/
2024.11.04 23:26:36.567870 [ 3698 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_27_112_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007fe94d0adac3
31. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:36.826256 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/eom/jkidxcqjrgbtdqnrdctnwxgsjgiup, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8789a2e5-a3e5-46e4-b117-0ac7d17c25dc, upload has finished false.
2024.11.04 23:26:36.826449 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/eom/jkidxcqjrgbtdqnrdctnwxgsjgiup, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8789a2e5-a3e5-46e4-b117-0ac7d17c25dc, upload has finished false
2024.11.04 23:26:36.848474 [ 3695 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/eom/jkidxcqjrgbtdqnrdctnwxgsjgiup, total size 117440512, count 117440512, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id 8789a2e5-a3e5-46e4-b117-0ac7d17c25dc, upload has finished false
2024.11.04 23:26:36.852242 [ 3695 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_3_3_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 47 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:37.807752 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_150_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column databases): (while reading from part /var/lib/clickhouse/disks/s3/store/8b5/8b5461f5-e764-40a9-b9e6-a69ed840495b/202411_1_122_21/ in table system.query_log (8b5461f5-e764-40a9-b9e6-a69ed840495b) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:250: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bbf82c
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:38.039504 [ 3694 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=114, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:38.396444 [ 3707 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=5). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
14. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
15. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
18. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
19. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
20. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
21. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
22. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
23. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
24. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
25. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
26. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
27. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
29. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
30. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:38.773498 [ 2929 ] {} <Warning> Application: Child process was stopped by signal 19.
2024.11.04 23:26:39.761240 [ 2929 ] {} <Warning> Application: Child process was continued.
2024.11.04 23:26:39.805579 [ 3693 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {8b5461f5-e764-40a9-b9e6-a69ed840495b::202411_1_150_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=50, jobs=36). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&, int&&) @ 0x0000000039723183
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000003971f3ff
11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000031c3deb3
12. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c65901
13. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:641: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031cfe010
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031bc08a8
15. ./build_docker/./src/DataTypes/Serializations/SerializationMap.cpp:0: DB::SerializationMap::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031c63e21
16. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
17. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x0000000039719b8b
18. ./contrib/llvm-project/libcxx/include/vector:537: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x000000003971952a
19. ./contrib/llvm-project/libcxx/include/__tree:1088: std::__unique_if<DB::MergeTreeReaderWide>::__unique_single std::make_unique[abi:v15000]<DB::MergeTreeReaderWide, std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) @ 0x00000000393a6d79
20. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003939af96
21. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
22. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
23. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
24. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
28. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
29. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
31. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:26:44.596503 [ 3694 ] {} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/mdo/yisgohqsbmejhjbiaicjjktjhiltd, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id bf778e73-939d-4f71-ba60-a14b34cf4ecc, upload has finished false.
2024.11.04 23:26:44.596866 [ 3694 ] {} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/mdo/yisgohqsbmejhjbiaicjjktjhiltd, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id bf778e73-939d-4f71-ba60-a14b34cf4ecc, upload has finished false
2024.11.04 23:26:44.644962 [ 3694 ] {} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/mdo/yisgohqsbmejhjbiaicjjktjhiltd, total size 234881024, count 234881024, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id bf778e73-939d-4f71-ba60-a14b34cf4ecc, upload has finished false
2024.11.04 23:31:46.869933 [ 8002 ] {920bf785-f6ce-4065-a997-e86b145ae6eb} <Error> executeQuery: Code: 521. DB::ErrnoException: Cannot rename /var/lib/clickhouse/metadata/test.sql.tmp to /var/lib/clickhouse/metadata/test.sql because the second path already exists: , errno: 17, strerror: File exists. (ATOMIC_RENAME_FAIL) (version 24.3.12.76.altinitystable (altinity build)) (from [::1]:56460) (in query: CREATE DATABASE IF NOT EXISTS test ), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::ErrnoException::ErrnoException<String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, String const&) @ 0x0000000031e45382
3. ./build_docker/./src/Common/atomicRename.cpp:90: DB::renameat2(String const&, String const&, int) @ 0x0000000031e44968
4. ./build_docker/./src/Common/atomicRename.cpp:211: DB::renameNoReplace(String const&, String const&) @ 0x0000000031e43604
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x0000000034605f4a
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
10. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850

2024.11.04 23:31:46.872485 [ 8002 ] {920bf785-f6ce-4065-a997-e86b145ae6eb} <Error> DynamicQueryHandler: Code: 521. DB::ErrnoException: Cannot rename /var/lib/clickhouse/metadata/test.sql.tmp to /var/lib/clickhouse/metadata/test.sql because the second path already exists: , errno: 17, strerror: File exists. (ATOMIC_RENAME_FAIL), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::ErrnoException::ErrnoException<String const&, String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String const&>::type>, String const&, String const&) @ 0x0000000031e45382
3. ./build_docker/./src/Common/atomicRename.cpp:90: DB::renameat2(String const&, String const&, int) @ 0x0000000031e44968
4. ./build_docker/./src/Common/atomicRename.cpp:211: DB::renameNoReplace(String const&, String const&) @ 0x0000000031e43604
5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x0000000034605f4a
6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::execute() @ 0x0000000034673234
7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000003659e465
8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::FormatSettings> const&, std::function<void (DB::IOutputFormat&)>) @ 0x00000000365ad21f
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000003a395be3
10. ./build_docker/./src/Server/HTTPHandler.cpp:1091: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x000000003a3a38e2
11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000003a546105
12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003e69db00
13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: Poco::Net::TCPServerDispatcher::run() @ 0x000000003e69e981
14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003eb25e86
15. ./build_docker/./base/poco/Foundation/src/Thread.cpp:46: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x000000003eb22c4e
16. ./base/poco/Foundation/src/Thread_POSIX.cpp:350: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003eb1fb32
17. ? @ 0x00007fe94d0adac3
18. ? @ 0x00007fe94d13f850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:32:00.890723 [ 2930 ] {} <Warning> Application: Closed all listening sockets. Waiting for 4 outstanding connections.
2024.11.04 23:32:05.796738 [ 2930 ] {} <Warning> Application: Closed connections. But 4 remain. Tip: To increase wait time add to config: <shutdown_wait_unfinished>60</shutdown_wait_unfinished>
2024.11.04 23:32:05.797272 [ 2930 ] {} <Warning> Application: Will shutdown forcefully.
2024.11.04 23:32:08.254481 [ 14084 ] {} <Warning> Context: Server was built with sanitizer. It will work slowly.
2024.11.04 23:32:08.270117 [ 14084 ] {} <Warning> Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production.
2024.11.04 23:32:08.273824 [ 14084 ] {} <Warning> Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl.
2024.11.04 23:32:08.275083 [ 14084 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.11.04 23:32:16.935907 [ 14084 ] {} <Warning> CloudPlacementInfo: Placement info has not been loaded
2024.11.04 23:32:17.137359 [ 14084 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.11.04 23:32:17.141589 [ 14084 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.11.04 23:32:17.141642 [ 14084 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.11.04 23:32:17.192767 [ 14762 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.11.04 23:32:18.921669 [ 14891 ] {} <Warning> system.metric_log (33f0dc8f-1542-43c0-a136-e415d11993b4): Removing temporary directory /var/lib/clickhouse/disks/s3/store/33f/33f0dc8f-1542-43c0-a136-e415d11993b4/tmp_insert_202411_181_181_0/
2024.11.04 23:32:18.921825 [ 14891 ] {} <Warning> system.metric_log (33f0dc8f-1542-43c0-a136-e415d11993b4): Since zero-copy replication is enabled we are not going to remove blobs from shared storage for /var/lib/clickhouse/disks/s3/store/33f/33f0dc8f-1542-43c0-a136-e415d11993b4/tmp_insert_202411_181_181_0/
2024.11.04 23:32:22.280269 [ 14084 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.11.04 23:32:25.741997 [ 14084 ] {} <Warning> loadMetadata: Removing temporary file /var/lib/clickhouse/metadata/test.sql.tmp
2024.11.04 23:32:26.753397 [ 14084 ] {} <Warning> TablesLoader: Tables test_13.test_dict depend on default.test_table, but seems like that does not exist. Will ignore it and try to load existing tables
2024.11.04 23:32:26.755674 [ 14084 ] {} <Warning> TablesLoader: Tables test_13.dist_01683 depend on default.n, but seems like that does not exist. Will ignore it and try to load existing tables
2024.11.04 23:32:26.755785 [ 14084 ] {} <Warning> TablesLoader: Tables test_13.dict depend on default.url, but seems like that does not exist. Will ignore it and try to load existing tables
2024.11.04 23:32:26.756151 [ 14084 ] {} <Warning> TablesLoader: Tables test_13.hashed_dictionary depend on default.dictionary_source_table, but seems like that does not exist. Will ignore it and try to load existing tables
2024.11.04 23:33:19.971508 [ 14878 ] {} <Error> void DB::SystemLog<DB::MetricLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::MetricLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::flushImpl(std::vector<DB::MetricLogElement, std::allocator<DB::MetricLogElement>> const&, unsigned long) @ 0x0000000035fc9d6b
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::MetricLogElement>::savingThreadFunction() @ 0x0000000035fc62e4
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::MetricLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f42f827
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f42f6ff
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:19.971668 [ 14792 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_113_178_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePart(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039885fe0
15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::MergeTask::MergeProjectionsStage::finalizeProjectionsAndWholeMerge() const @ 0x000000003906abfc
16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:840: DB::MergeTask::MergeProjectionsStage::execute() @ 0x000000003906b0c4
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
18. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
19. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
21. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
22. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
23. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
25. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
27. ? @ 0x00007f47e248fac3
28. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:20.076658 [ 14970 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:47804 -> [::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000001f5109df
3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001f50f353
4. ./src/IO/WriteBuffer.h:65: Coordination::ZooKeeper::flushWriteBuffer() @ 0x000000003c13cefe
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:709: Coordination::ZooKeeper::sendThread() @ 0x000000003c153713
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b252
7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
9. ? @ 0x00007f47e248fac3
10. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:20.077118 [ 14403 ] {} <Warning> test_13.alter_compression_codec1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:33:20.077163 [ 14404 ] {} <Warning> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:33:20.077194 [ 14405 ] {} <Warning> test_13.table_rename_with_ttl (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:33:20.077283 [ 14406 ] {} <Warning> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:33:20.077363 [ 14407 ] {} <Warning> test_13.t3 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:33:20.077455 [ 14408 ] {} <Warning> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:33:20.077594 [ 14409 ] {} <Warning> test_13.alter_compression_codec2 (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session.
2024.11.04 23:33:20.240059 [ 14858 ] {} <Error> ZooKeeperClient: Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. DB::Exception::Exception<unsigned long&, String>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<String>::type>, unsigned long&, String&&) @ 0x0000000008f26d79
3. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x0000000008f269cd
4. ./src/Common/transformEndianness.h:18: Coordination::ZooKeeper::receiveEvent() @ 0x000000003c1569eb
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::receiveThread() @ 0x000000003c155719
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b912
7. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
9. ? @ 0x00007f47e248fac3
10. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:21.011823 [ 14876 ] {} <Error> void DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TransactionsInfoLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988eefc
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::flushImpl(std::vector<DB::TransactionsInfoLogElement, std::allocator<DB::TransactionsInfoLogElement>> const&, unsigned long) @ 0x0000000036076f2a
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TransactionsInfoLogElement>::savingThreadFunction() @ 0x0000000036073688
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TransactionsInfoLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f438e67
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f438d3f
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007f47e248fac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:21.819169 [ 14889 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=3). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:22.014015 [ 14970 ] {} <Error> ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:0: Poco::IOException::IOException(String const&, int) @ 0x000000003e9ff82e
2. ./build_docker/./base/poco/Net/src/NetException.cpp:0: Poco::Net::NetException::NetException(String const&, int) @ 0x000000003e668a8e
3. ./build_docker/./base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x000000003e685f7a
4. ./contrib/llvm-project/libcxx/include/string:1499: Poco::Net::SocketImpl::shutdown() @ 0x000000003e68aeb9
5. ./build_docker/./base/poco/Net/src/StreamSocket.cpp:109: Poco::Net::StreamSocket::shutdown() @ 0x000000003e695dee
6. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1040: Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000003c13ebb8
7. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::sendThread() @ 0x000000003c15401a
8. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000003c16b252
9. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
11. ? @ 0x00007f47e248fac3
12. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:22.902054 [ 14789 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/mgq/jorliwenjxduyzmkfshlqyogfwmmi, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a899b007-8c22-49b3-b110-581938552cb2, upload has finished false.
2024.11.04 23:33:22.902159 [ 14789 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/mgq/jorliwenjxduyzmkfshlqyogfwmmi, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a899b007-8c22-49b3-b110-581938552cb2, upload has finished false
2024.11.04 23:33:22.951755 [ 14789 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/mgq/jorliwenjxduyzmkfshlqyogfwmmi, total size 150994944, count 150994944, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id a899b007-8c22-49b3-b110-581938552cb2, upload has finished false
2024.11.04 23:33:23.640390 [ 14865 ] {} <Error> void DB::SystemLog<DB::QueryLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::QueryLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./contrib/llvm-project/libcxx/include/vector:1595: DB::IMergeTreeDataPart::MinMaxIndex::store(std::vector<String, std::allocator<String>> const&, std::vector<std::shared_ptr<DB::IDataType const>, std::allocator<std::shared_ptr<DB::IDataType const>>> const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e82aec
13. ./contrib/llvm-project/libcxx/include/vector:434: DB::IMergeTreeDataPart::MinMaxIndex::store(DB::MergeTreeData const&, DB::IDataPartStorage&, DB::MergeTreeDataPartChecksums&) const @ 0x0000000038e7f509
14. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x000000003988cd63
15. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
18. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
19. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
22. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
23. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
26. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::flushImpl(std::vector<DB::QueryLogElement, std::allocator<DB::QueryLogElement>> const&, unsigned long) @ 0x000000003600ad8a
28. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::QueryLogElement>::savingThreadFunction() @ 0x00000000360074e8
29. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::QueryLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f433007
30. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f432edf
31. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:23.861639 [ 14343 ] {} <Error> test_13.`03014_async_with_dedup_part_log` (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/table/test_13/03014_async_with_dedup_part_log/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007f47e248fac3
16. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:23.862250 [ 14340 ] {} <Error> test_13.t3 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Session expired, path /02946_parallel_replicas/test_13/test_tbl/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007f47e248fac3
16. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:23.862339 [ 14341 ] {} <Error> test_13.test_03217_system_tables_replica_1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Session expired, path /clickhouse/tables/test_13/test_03217_system_tables_replica/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007f47e248fac3
16. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:23.862407 [ 14342 ] {} <Error> test_13.alter_compression_codec2 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Session expired, path /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007f47e248fac3
16. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:23.862472 [ 14346 ] {} <Error> test_13.test_03217_system_tables_replica_2 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Session expired, path /clickhouse/tables/test_13/test_03217_system_tables_replica/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007f47e248fac3
16. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:23.862501 [ 14345 ] {} <Error> test_13.alter_compression_codec1 (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Session expired, path /clickhouse/tables/test_13/test_00910/test_13alter_compression_codecs/s1/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007f47e248fac3
16. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:23.862666 [ 14344 ] {} <Error> test_13.table_rename_with_ttl (ReplicatedMergeTreeCleanupThread): void DB::ReplicatedMergeTreeCleanupThread::run(): Code: 999. Coordination::Exception: Coordination error: Session expired, path /clickhouse/test_13/test/table_rename_with_ttl_01378/log. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const*, String const&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String const&>::type>, char const*&&, String const&) @ 0x000000002fbce119
3. ./src/Common/ZooKeeper/IKeeper.h:490: Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x000000002fbcbdc0
4. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::existsWatch(String const&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x000000003c045c80
5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:498: zkutil::ZooKeeper::exists(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x000000003c03cdb1
6. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:203: DB::ReplicatedMergeTreeCleanupThread::clearOldLogs() @ 0x0000000039a837f7
7. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::iterate() @ 0x0000000039a8109f
8. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp:0: DB::ReplicatedMergeTreeCleanupThread::run() @ 0x0000000039a7d7b2
9. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000039aafcff
10. ./src/Common/Stopwatch.h:24: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x0000000030ff07e1
11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x0000000030ffb80e
12. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000030ffd432
13. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
15. ? @ 0x00007f47e248fac3
16. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:24.354297 [ 14789 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_4_4_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 16 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:24.354312 [ 14794 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {386c9b4d-67f8-4db6-a90a-ff6424edf355::202411_113_179_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::future<std::shared_ptr<DB::MarksInCompressedFile>> std::__function::__policy_invoker<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>::__call_impl<std::__function::__default_alloc_func<std::function<std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<std::shared_ptr<DB::MarksInCompressedFile>, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority), std::future<std::shared_ptr<DB::MarksInCompressedFile>> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)>>(std::__function::__policy_storage const*, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority&&) @ 0x0000000039664b6a
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000003965b2b7
8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:0: DB::MergeTreeMarksLoader::MergeTreeMarksLoader(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::MarkCache*, String const&, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool, DB::ReadSettings const&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long) @ 0x000000003965a32b
9. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr<DB::MergeTreeMarksLoader> std::allocate_shared[abi:v15000]<DB::MergeTreeMarksLoader, std::allocator<DB::MergeTreeMarksLoader>, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String, unsigned long, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*, unsigned long, void>(std::allocator<DB::MergeTreeMarksLoader> const&, std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>&, DB::MarkCache* const&, String&&, unsigned long&&, DB::MergeTreeIndexGranularityInfo const&, bool&, DB::ReadSettings&, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>*&&, unsigned long&&) @ 0x00000000396e8143
10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr<DB::IMergeTreeDataPartInfoForReader>, DB::NamesAndTypesList, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>>, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int) @ 0x00000000396daed4
11. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:0: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer<std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int>(std::shared_ptr<DB::LoadedMergeTreeDataPartInfoForReader>&, DB::NamesAndTypesList const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&, int&&) @ 0x00000000393872d4
12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr<DB::StorageSnapshot> const&, DB::MarkRanges const&, std::unordered_map<String, DB::Field, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, DB::Field>>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr<DB::AlterConversions const> const&, DB::MergeTreeReaderSettings const&, std::map<String, double, std::less<String>, std::allocator<std::pair<String const, double>>> const&, std::function<void (DB::ReadBufferFromFileBase::ProfileInfo)> const&) const @ 0x000000003937f2ac
13. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:177: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool) @ 0x000000003972a854
14. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x00000000397338b5
15. ./contrib/llvm-project/libcxx/include/__memory/allocation_guard.h:66: std::shared_ptr<DB::MergeTreeSequentialSource> std::make_shared[abi:v15000]<DB::MergeTreeSequentialSource, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>, bool, bool&, bool&, bool&, void>(DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>&, std::vector<String, std::allocator<String>>&, std::optional<DB::MarkRanges>&&, bool&&, bool&, bool&, bool&) @ 0x0000000039732539
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:312: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr<DB::StorageSnapshot> const&, std::shared_ptr<DB::IMergeTreeDataPart const>, std::vector<String, std::allocator<String>>, std::optional<DB::MarkRanges>, bool, bool, bool, bool, std::shared_ptr<std::atomic<unsigned long>>) @ 0x0000000039730f8d
17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000003904c1b5
18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000003904009d
19. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
20. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
21. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
22. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
23. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
24. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
25. ./src/Common/Stopwatch.h:26: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000001f34d004
26. ./build_docker/./src/Common/ThreadPool.cpp:0: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x000000001f3595f3
27. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f35949f
28. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
30. ? @ 0x00007f47e248fac3
31. ? @ 0x00007f47e2521850
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:25.065583 [ 14884 ] {} <Error> void DB::SystemLog<DB::TraceLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TraceLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr<DB::IMergeTreeDataPart> const&, DB::MergeTreeDataPartChecksums&) @ 0x00000000398903b1
13. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x0000000039888eea
14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
18. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
21. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
24. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
25. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::flushImpl(std::vector<DB::TraceLogElement, std::allocator<DB::TraceLogElement>> const&, unsigned long) @ 0x000000003606156b
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TraceLogElement>::savingThreadFunction() @ 0x000000003605df49
27. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TraceLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f437b27
28. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f4379ff
29. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
31. ? @ 0x00007f47e248fac3
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:25.141531 [ 14889 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) [LogElement = DB::TextLogElement]: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=6). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: std::function<std::future<void> (std::function<void ()>&&, Priority)> DB::threadPoolCallbackRunnerUnsafe<void, std::function<void ()>>(ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>&, String const&)::'lambda'(std::function<void ()>&&, Priority)::operator()(std::function<void ()>&&, Priority) @ 0x0000000030efd87f
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000030efcf80
8. ./contrib/llvm-project/libcxx/include/future:1237: DB::TaskTracker::add(std::function<void ()>&&) @ 0x0000000030460892
9. ./build_docker/./src/IO/WriteBufferFromS3.cpp:0: DB::WriteBufferFromS3::makeSinglepartUpload(DB::WriteBufferFromS3::PartData&&) @ 0x0000000030f8c9dd
10. ./build_docker/./src/IO/WriteBufferFromS3.cpp:181: DB::WriteBufferFromS3::preFinalize() @ 0x0000000030f8b0c5
11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:0: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x000000003247248f
12. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:297: DB::MergeTreeDataPartWriterCompact::fillDataChecksums(DB::MergeTreeDataPartChecksums&) @ 0x00000000393b1c8c
13. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterCompact.cpp:420: DB::MergeTreeDataPartWriterCompact::fillChecksums(DB::MergeTreeDataPartChecksums&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>&) @ 0x00000000393b3c45
14. ./contrib/llvm-project/libcxx/include/__hash_table:1727: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr<DB::IMergeTreeDataPart> const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*) @ 0x000000003988643a
15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:0: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>, long, bool) @ 0x0000000039967c12
16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:404: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::Context const>) @ 0x000000003995bc0a
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk) @ 0x0000000039dc6aaa
18. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:0: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000003b2f7130
19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::ExceptionKeepingTransform::work()::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000003affdbe0
20. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function<void ()>, DB::ThreadStatus*, std::atomic<unsigned long>*) @ 0x000000003affd74e
21. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:0: DB::ExceptionKeepingTransform::work() @ 0x000000003affcb6c
22. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
25. ./build_docker/./src/Processors/Executors/PushingPipelineExecutor.cpp:123: DB::PushingPipelineExecutor::finish() @ 0x000000003a6b5ed9
26. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::flushImpl(std::vector<DB::TextLogElement, std::allocator<DB::TextLogElement>> const&, unsigned long) @ 0x00000000360b7c6a
27. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog<DB::TextLogElement>::savingThreadFunction() @ 0x00000000360b43c8
28. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()>(DB::SystemLogBase<DB::TextLogElement>::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001f43c2e7
29. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000001f43c1bf
30. ./base/base/../base/wide_integer_impl.h:810: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000001f343a95
31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:303: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000001f351b8b
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:26.552984 [ 14797 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: WriteBufferFromS3 was neither finished nor aborted, try to abort upload in destructor. Details: bucket test, key test/cdw/vvbeocbpudwipvlubqblmowxcovdv, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ee14fe0d-6de7-4cf7-81f4-a7fb1b553db4, upload has finished false.
2024.11.04 23:33:26.553096 [ 14797 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Abort multipart upload. Details: bucket test, key test/cdw/vvbeocbpudwipvlubqblmowxcovdv, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ee14fe0d-6de7-4cf7-81f4-a7fb1b553db4, upload has finished false
2024.11.04 23:33:26.558627 [ 14797 ] {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1} <Warning> WriteBufferFromS3: Multipart upload has aborted successfully. Details: bucket test, key test/cdw/vvbeocbpudwipvlubqblmowxcovdv, total size 50331648, count 50331648, hidden_size 0, offset 0, with pool: true, prefinalized false, finalized false, upload id ee14fe0d-6de7-4cf7-81f4-a7fb1b553db4, upload has finished false
2024.11.04 23:33:26.790405 [ 14084 ] {} <Error> virtual void DB::StorageBuffer::flushAndPrepareForShutdown(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x00000000387d34a8
9. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAndPrepareForShutdown() @ 0x00000000387d2e4f
10. ./contrib/llvm-project/libcxx/include/__tree:189: DB::DatabaseWithOwnTablesBase::shutdown() @ 0x0000000031f66c83
11. ./build_docker/./src/Databases/DatabaseOnDisk.cpp:179: DB::DatabaseOnDisk::shutdown() @ 0x0000000031dd5eca
12. ./contrib/llvm-project/libcxx/include/__tree:189: DB::DatabaseCatalog::shutdownImpl() @ 0x0000000032b6f920
13. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:865: DB::DatabaseCatalog::shutdown() @ 0x0000000032b88e0f
14. ./build_docker/./src/Interpreters/Context.cpp:0: DB::ContextSharedPart::shutdown() @ 0x0000000032987ace
15. ./build_docker/./src/Interpreters/Context.cpp:4446: DB::Context::shutdown() @ 0x000000003295c3bf
16. ./build_docker/./programs/server/Server.cpp:0: DB::Server::main(std::vector<String, std::allocator<String>> const&)::$_1::operator()() const @ 0x000000001f653f26
17. ./base/base/../base/scope_guard.h:0: DB::Server::main(std::vector<String, std::allocator<String>> const&) @ 0x000000001f618565
18. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x000000003e6d791f
19. ./build_docker/./programs/server/Server.cpp:419: DB::Server::run() @ 0x000000001f5e6deb
20. ./build_docker/./base/poco/Util/src/ServerApplication.cpp:132: Poco::Util::ServerApplication::run(int, char**) @ 0x000000003e721200
21. ./build_docker/./programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0x000000001f5df349
22. ./build_docker/./programs/main.cpp:0: main @ 0x0000000007c51bf5
23. ? @ 0x00007f47e2424d90
24. ? @ 0x00007f47e2424e40
25. _start @ 0x0000000007bcc0ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:26.790627 [ 14797 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {d0d49101-dd5d-4ded-b328-98d83df5ff93::all_1_9_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=0): (while reading column Title): (while reading from part /var/lib/clickhouse/disks/s3/store/d0d/d0d49101-dd5d-4ded-b328-98d83df5ff93/all_2_2_0/ in table test_7.hits_none (d0d49101-dd5d-4ded-b328-98d83df5ff93) located on disk s3 of type s3, from mark 18 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002ff4ffea
7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002ff4c653
8. ./src/IO/AsynchronousReader.h:48: DB::AsynchronousBoundedReadBuffer::readAsync(char*, unsigned long, Priority) @ 0x00000000303e811a
9. ./build_docker/./src/Disks/IO/AsynchronousBoundedReadBuffer.cpp:0: DB::AsynchronousBoundedReadBuffer::prefetch(Priority) @ 0x00000000303e8c87
10. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:0: non-virtual thunk to DB::CompressedReadBufferFromFile::prefetch(Priority) @ 0x00000000316ed1ee
11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker<void (DB::ISerialization::SubstreamPath const&)>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x0000000039721cf7
12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:63: DB::ISerialization::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function<void (DB::ISerialization::SubstreamPath const&)> const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000031b9f314
13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::enumerateStreams(std::function<void (DB::ISerialization::SubstreamPath const&)> const&, std::shared_ptr<DB::IDataType const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&) const @ 0x0000000031b9feaf
14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::prefetchForColumn(Priority, DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, unsigned long, bool, unsigned long, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>&) @ 0x000000003971b3a3
15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:103: DB::MergeTreeReaderWide::prefetchForAllColumns(Priority, unsigned long, unsigned long, unsigned long, bool) @ 0x000000003971ac80
16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000003971ba9e
17. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:227: DB::MergeTreeSequentialSource::generate() @ 0x000000003972dba4
18. ./src/Processors/Chunk.h:90: DB::ISource::tryGenerate() @ 0x000000003a635d9c
19. ./contrib/llvm-project/libcxx/include/optional:344: DB::ISource::work() @ 0x000000003a63552c
20. ./contrib/llvm-project/libcxx/include/list:588: DB::ExecutionThreadContext::executeTask() @ 0x000000003a69cf57
21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000003a66a6f8
22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:150: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000003a669404
23. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000003a6aacf2
24. ./build_docker/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000003a6aaf3f
25. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:450: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000039057b08
26. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:438: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x00000000390578c4
27. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000003906a02c
28. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000039dccfc0
29. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:294: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x00000000390ae1eb
30. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:0: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000390b09ee
31. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000390b7aff
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:27.192117 [ 14084 ] {} <Error> virtual void DB::StorageBuffer::flushAndPrepareForShutdown(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003e9efbc9
1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001f112a13
2. ./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000001f351310
3. ./build_docker/./src/Common/ThreadPool.cpp:203: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001f35806e
4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000001f346e0b
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001f345b4a
6. ./src/Common/threadPoolCallbackRunner.h:0: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>, std::function<void ()>>::operator()(std::function<void ()>&&, Priority) @ 0x000000002fa9255d
7. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAllBuffers(bool) @ 0x00000000387d3f02
8. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::optimize(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::IAST> const&, bool, bool, std::vector<String, std::allocator<String>> const&, bool, std::shared_ptr<DB::Context const>) @ 0x00000000387d34a8
9. ./build_docker/./src/Storages/StorageBuffer.cpp:0: DB::StorageBuffer::flushAndPrepareForShutdown() @ 0x00000000387d2e4f
10. ./src/Storages/IStorage.h:580: DB::DatabaseWithOwnTablesBase::shutdown() @ 0x0000000031f66eb4
11. ./build_docker/./src/Databases/DatabaseOnDisk.cpp:179: DB::DatabaseOnDisk::shutdown() @ 0x0000000031dd5eca
12. ./contrib/llvm-project/libcxx/include/__tree:189: DB::DatabaseCatalog::shutdownImpl() @ 0x0000000032b6f920
13. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:865: DB::DatabaseCatalog::shutdown() @ 0x0000000032b88e0f
14. ./build_docker/./src/Interpreters/Context.cpp:0: DB::ContextSharedPart::shutdown() @ 0x0000000032987ace
15. ./build_docker/./src/Interpreters/Context.cpp:4446: DB::Context::shutdown() @ 0x000000003295c3bf
16. ./build_docker/./programs/server/Server.cpp:0: DB::Server::main(std::vector<String, std::allocator<String>> const&)::$_1::operator()() const @ 0x000000001f653f26
17. ./base/base/../base/scope_guard.h:0: DB::Server::main(std::vector<String, std::allocator<String>> const&) @ 0x000000001f618565
18. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x000000003e6d791f
19. ./build_docker/./programs/server/Server.cpp:419: DB::Server::run() @ 0x000000001f5e6deb
20. ./build_docker/./base/poco/Util/src/ServerApplication.cpp:132: Poco::Util::ServerApplication::run(int, char**) @ 0x000000003e721200
21. ./build_docker/./programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0x000000001f5df349
22. ./build_docker/./programs/main.cpp:0: main @ 0x0000000007c51bf5
23. ? @ 0x00007f47e2424d90
24. ? @ 0x00007f47e2424e40
25. _start @ 0x0000000007bcc0ee
 (version 24.3.12.76.altinitystable (altinity build))
2024.11.04 23:33:29.034321 [ 14764 ] {} <Error> RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled